We gather and report loads of data nearly every day.
Is your data “good data”? Or does it fall into the “bad data” category?
Let’s define the difference between good and bad data. Good data is accurate, timely, and useful. Bad data is not. It may be time to look at each set of data you are collecting or reviewing and judge if it’s good or not. Then set plans in motion to minimize the presence of bad data in your organization.
Good data is accurate
By this I mean it truly reprints the items or process being measured.
If the mass is 2.3 kilograms, then the measurement should be pretty close to 2.3 kg. This is a basic assumption we make when reviewing measurements, yet when was the last time you checked? Use a different measurement method, possible a known accurate method to check.
Measurement system analysis includes a few steps to determine if the gage making a measurement is true or not. Calibration may come to mind, as it is a step to verify the gage readings are reflecting standard measures. A meter is a meter is a meter across the many ways we can measure distance.
It also includes checking the common sources of measurement error:
You may also want to understand the resolution or discrimination of the measurement process.
If these terms and how one goes about checking for accuracy, it may be time to learn a little about MSA.
Good data is timely
If the experiment results are available a week after the decision to launch the product, it will not be considered in the decision. It is not useful for the decision concerning product launch. If the data was available it may alter the decision. Late, we will not know.
Timely means it is in time for someone or some team to make a decision. Ideally, the data is available immediately. When a product fails in the field, we would like to know right away, not two or three month later. If a production line becomes unstable, knowing before another unit of scrap is produced would be timely.
Not all data gathering and reporting is immediate. Some data takes months or an entire year to gather. There are physical constraints in some situation that day the gathering of data. For example is takes on average 13 minutes, 48 seconds, for radio signals to travel from a space probe orbiting Mars to reach Earth . If you are making important measurements on Earth it should be a shorter delay.
The key point here, is the data should be available when it is needed to make decisions.
Good data is useful
Even if the data is accurate and timely is may not be useful. The data could be from a perfect measurement process, yet is measuring something we do not need to know or consider. The data gathered does not help inform us concerning the decision at hand.
For example, if I’m perfectly measuring production throughput, it does not help me understand the causes of the product line downtime. While related to some degrees, instead of the tally of units produced per hour, what we really would find useful is data concerning the number of interruptions to production, plus details on the root cause of each.
Setting up and maintaining the important measurements is difficult as we often shift focus based on the current data. We spot a trend and want to learn more than the current data can provide. The idea is we should not setup and only use a fixed set of data collection processes. Ideally your work to gather data is driven by the need to answer questions.
Is the maintenance process improving the equipment operation?
Is our manufacturing process stable and capable of creating our product?
Will the current product design meet life expectations/requirements?
Have we confirmed the new design ‘fixed’ the faults seen in the last prototype?
We have questions and we gather data to allow us to answer questions.
How would you describe the data you will look at today? Good or Bad? And more importantly, do you know if your data is good or bad?
A result of life testing can be measurement or evaluation of the lifetime.
Measurement of the lifetime requires a lot of testing to failure. The results provide us with the life (time-to-failure) distribution of the product itself. It is long and expensive.
Evaluation of the lifetime does not require as many test samples and these tests can be without failures. It is faster and cheaper . A drawback of the evaluation is that it does not give us the lifetime distribution. The evaluation checks the lower bound of reliability only, and interpretation of the results depends on the method of evaluation (the number of samples, test conditions, and the test time). Continue reading Lifetime Evaluation vs. Measurement. Part 2.→
One of the issues I’ve had with failure modes and effects analysis is the focus on failure modes.
The symptoms that the customer or end user will experience are important. If a customer detects that product has failed, that is a failure. The FMEA process does help us to identify and focus on the important elements of a design that improve the product reliability. That is all good.
The issue is the FMEA process doesn’t go far enough to really aide the team focus on what action to take when addressing a failure mode. The process does include the discussion of causes of the failure mode. The causes are often the team members educated opinions on what is likely to cause the failure mode. Often the description of the a cause is a failed part, faulty code, or faulty assembly.
Generally the discussion of causes is vague.
Failure Mechanisms versus Failure Modes
Failures modes are best described as what the customer experiences (no power, loss of function, etc.). Failure mechanisms are the root physical or chemical anomaly that leads to the existence of the failure mode. While we want to remove failure modes, we have to solve, remove, or mitigate failure mechanisms along the way.
The traditional FMEA process in my experience often provides vague classes of causes, hints at potential failure mechanisms, or avoids specifying mechanisms entirely. The actions items from the FMEA study then include investigations to find and understand the actual failure mechanisms (at best) or attempt to address vague classes of mechanisms with broad sweeps of monitoring, testing, or design changes.
Instead focusing the discussion on causes of failures at the level of failure mechanisms, enhances the discussion. Instead of talking about the causes as a component failure, it changes to what happens such that the component fails. Instead a vague average failure rate, it becomes a discussion about design or process errors or variation that leads to the components demise.
The hard part of this approach is the sheer number of ways (root causes) that an item may fail. Consider a simple component solder joint. The potential root causes includes:
And many others potential issues. Even these brief descriptions may have underlying causes which are the elements requiring attention in order to solve.
Fault Tree Analysis (FTA) and FMEA
Detailing all possible root causes of each failure mode would be tedious and I would suggest unnecessary. One approach I’ve seen is the common approach to FMEA, where we explore the class or basic expected types of root causes that lead to the listed failure mode. Then for the lines in the FMEA study that percolate to the items requiring attention, we then conduct a detailed FTA that flushes out the range and relative frequency of occurrence of the many different underlying failure mechanisms that lead to a specific failure mode.
If the primary cause of a failure mode is a faulty component, then what are the specific mechanisms that lead to a component being faulty. FTA is the right tool here. Used on conjunction with the highest risks identified in the FMEA permit the team to understand and solve or mitigate the right elements in the design or process to make a difference. Being specific with actions that make a difference is the key.
With your work to identify and resolve risks to reliability performance, how do you insure the solutions are actually solving the right problem? What works for you in your organization? How do you extend your FMEA work into effective action?
Having the technical and business skills is not sufficient to be a good reliability engineer.
You must also work with other people. With your peers, across the management team, with suppliers, contractors, and customers.
The ability to work well with others is often complex and situational. Being aware of a few basic skills will allow you learn and improve. Prette and Prette define social competence as the social skills
that meet the different inter-personal demands in the workplace in order to achieve the goals, preserve the well-being of the staff and respect the rights of each other.
A. Del Prette and Z. A. P. Del Prette, Psicologia das relac ̧o ̃es interpessoais: viveˆncias para o trabalho em grupo, Vozes, Petro ́ polis, 2001.
An engineer needs an awareness of the social situation and how their behavior influences others, along with a capability to correctly understand the behavior and needs of others. The concepts discussed under emotional intelligence include:
and social skills
Goleman, D., Emotional Intelligence: Why It Can Matter More than IQ. New York: Bantam Books (1995).
Goleman, D., Working with Emotional Intelligence. London: Bloomsbury Publishing (1998).
The ability to influence others or to aide in understanding a technical situation relies on effective communication. Beyond presenting the facts, finding, and conclusions, your communication must also build upon the audiences’ current understanding and capability. Also, our presentation must address the needs and expectations of the audience. The audience needs are often unstated, thus the need for your ability to correctly assess the social situation.
A Meeting Example
Let’s say two engineers join a meeting to discuss an engineering problem that requires a solution. One engineer, Juan, has social skills and the other, Tomas, does not. As the team assemblies Juan arrives a minute or two early and greets his co-workers and responds to greetings and comments pleasantly. Tomas arrives on time, does not greet anyone and focuses on his laptop catching up on a few emails messages till the meeting is called to order.
A member of the team opens with a short review of the specific technical challenge and as she started with the review of what is known to date, is interrupted by Tomas. Tomas launches into his solution for the problem and remarks that the remainder of the meeting is pointless as he already has an appropriate solution to implement. The solution is not obvious to the remainder of the group which frustrates Tomas as he repeats his assertion that he has a solution. There is social tension building which Tomas does not recognize.
Juan does sense the discontent between Tomas and the rest of the team. Juan does not fully understand the problem nor Tomas’ solution, yet injects a few questions that help guide Tomas to guide the team to better understand the problem and proposed solution. Juan facilitates a discussion between those with knowledge of the problem so the entire team fully understands the issue, plus assists Tomas restate the proposed solution again to help everyone understand the proposal.
The story of this meeting could continue, with more elements of lack of social awareness and skills and possible methods to create progress. It is situations without someone like Juan having the awareness and social adeptness to facilitate effective and socially acceptable communication, that likely end poorly.
Finding a solution is not the only goal of a problem solving meeting. It is finding a solution that the team can implement effectively. This requires the team understand both the problem and the solution. Furthermore, the one meeting is likely part of series of regular engagements for this team, thus impacts the ability of the team to work cohesively going forward.
When social behavior elements such as discussions, questions, and proposals, for example, do not include consideration of the recipients situation that social friction occurs. Those around Tomas may feel belittled, devalued, excluded or ignored. Those around Juan feel heard, included, and accepted. The ability of Juan to adjust his behavior based on his awareness permits the team to hear and understand any proposal, including those from social inept people.
To make a potentially long story short, even is Tomas had the correct solution for the problem, it was Juan’s people skills that permitted the team to find and implement a solution. Juan will likely advance in his career while Tomas will not.
If you are not familiar with emotional intelligence you may want to read a few introductory articles. If you have not examined your ability communicate with individuals or groups, or you have wondered how to improve your ability to influence those around you, look for material (articles, books, seminars, courses) that provide a framework to understand how to communicate effectively.
When I first became an engineer the focus on my education and training focused on technical skills. Later I learned the importance of social skills and found that my technical prowess (little that it was) flourished as others were able to understand and accept my ideas and solutions. Plus, my contributions to meetings helped with the acceptance of other better solutions.
There are a lot of talented people all around us. Our ability to work with them enhances our ability to implement engineering solutions that meet our business and customer expectations. Plus, working well with our team make our work a bit more enjoyable.
Having a master of business administration (MBA) may be helpful it is not required, yet knowing the warranty and brand cost per failure is essential.
You also need to know which analysis to conduct and how it fits into the larger program, organization, and how it impacts your customers. You need to know the business side of your work as well.
You need to understand the business connection as you create plans and finalize analysis. Each test proposed should include a business connection. Each improvement proposed has to balance with the other priorities and objectives and remain compelling.
What is Important to Your Organization?
I don’t know as I write this what is important to your organization today. It varies, even within an organization and over time. You do need to know what is important and why to become a good reliability engineer.
It is not enough to do what someone tells you to do. At first that may get you starting yet how you connect your work to adding value become essential. Practice finding the connection with every task.
When do you need this information?
When will this need an initial review?
When do you need a budget estimate?
Who will review this report?
Who will need this information?
What level of reliability knowledge does the audience have?
What level of detail is necessary?
And, so on…
The idea is to find who needs what by when and why?
This helps you plan, focus and deliver what is considered important at the moment in your organization.
For a high volume consumer product sold during the holiday period, time to market may be the top priority. This is because a delay to shipping means the loss of sales for the year. Missing the deadline is not an option. Thus, your work as a reliability engineer has to focus on how you can minimize the risk to any delays.
Now, you know that finding a critical reliability issue late in the process may delay the program, therefore focusing on reducing the chance of a late discovery is your focus. Early testing and development work on the highest risk element may help you identify reliability risks that have a long lead time to test and evaluate.
For a industrial product with relatively low volume the time to market pressure is not as intense. In this case, the brand image may be paramount. Thus getting the initial units as reliability as customer expect becomes an overriding priority.
One of the issues with this situation is the lack of prototype systems to evaluate. You may not have the capability to test full systems in any quantity prior to to the start of production, if ever. Thus, you work to identify and characterize each potential failure mechanisms becomes critical. The ability to create a viable system model allows you to prioritize your work on improving the critical few elements that will impact system reliability performance.
Other priorities may focus on cost reduction, cutting edge feature sets, or ease of use. Your organization likely has a clear priority. It’s then up to you to connect your work in reliability engineering to that priority.
Now Connect Each Reliability Task to the Value it Provides
For each reliability task proposed or delivered what is the value it provides connected to the primary priority (or secondary priority) in a clear and concise way. If the task or activity doesn’t add value, why are you doing it?
In the high volume consumer product situation, reducing the risk of a shipment delay provide value. Let’s say you are proposing HALT early in the program. How would that add value? By potentially identifying design or manufacturing faults if discovered late in the program would delay the project, then you arguably reduce the risk of delays by some amount.
For the low volume complex equipment situation, a HALT would improve the ability of the team to find and solve complex issues that may otherwise go undetected. Again, HALT would add value. In each case there is not a hard and fast formula yet there is a clear step by step connection between the task and the value to the organization.
There are two steps here. One understand the business and customers to the extent that you clearly understand what is important. Second, articulate the value related to the business priorities for each reliability task or activity.
How can we tell whether an iron is hot enough? The answer is obvious: We can measure temperature by using a thermocouple and a meter. But, in practice, we lick our finger and touch the iron. Sizzle…. Yes, it’s hot!
The Technical Skills of a Good Reliability Engineer
The fundamental technical skills, as I see it, have to include statistics and root cause analysis skills. This skill set is one of three broad areas introduced in the article, What Makes the Best Reliability Engineer?
I would say these are the minimum technical skills for a good reliability engineer. Able to calculate sample size requirements, understand a dataset, and correctly determine the root causes of a failure.
There are others skills that would be great to include, such as electrical, mechanical and software engineering, plus materials science, physics, and chemistry. Yet, what separates a good reliability engineer from other types of engineering is our ability to plan and analyze life tests and to truly understand how and why failures occur.
This is often considered the same as leaping tall buildings with a single bound with respect to skill level.
Few enjoyed their undergraduate statistics class and recently fewer campuses require a stats course. Statistics is the language of variation and is essential for our understanding of the world our products experience.
If every product met the exact specifications of the design and only operated in one set of environmental and use conditions, we would have fewer field failures. If every failure mechanism led to failure exactly the same way within each and every product, we would have far fewer field failures.
Variability may lead to elements of a product being out of spec, or drifting/wearing to an out of spec conditions, thus failing. Variability may also lead to changes in the stress/strength relationships, again increasing the number of failures over time.
The ability of a good reliability engineer to use available data and statistical techniques to:
Estimate sample size requirements for environmental testing
Analyze vendor life testing results
Summarize field failure and warranty datasets
Is just the start of our expected statistical prowess. We also need statistical skills to:
Monitor and control processes
Design and analyze screening and optimization design of experiments
Review and identify field failure trends and unique failure mechanisms
Your ability to use the right tool to quickly solve a problem may span statistical process control, hypothesis testing, regression analysis, and life data analysis all before noon. That may well be like stopping a speeding bullet level of skill.
You may need to master all these elements of statistics if you’re working as a lone reliability engineer, or rely on a trusted colleague is so fortunate. Either way you need to understand enough statistics to know when and how to apply this set of technical skills.
Root Cause Analysis
Failure mechanisms are hard science – even the human factors related failures. Failures occur because something occurs at an atomic, molecular, code or interaction level that precipitate an error or fault to manifest.
Your technical skill includes understanding the range of possible errors and faults that may occur with your product and how to avoid, minimize or mirage each one. It may not be possible to anticipate and fully understand every possible failure mechanisms, thus we focus on the most likely and common, plus continue to learn about those new (or interesting) failure mechanisms that appear.
A second element to this set of skills is the ability to deduce the root cause of a failure. Given a failure, you should be able to conduct the root causes analysis to determine the underlying failure mechanism and initiating circumstance. This permits the team to take corrective action that actually works.
The skill set includes
Gathering evidence and understanding the relationships and contributing factors
Delving into the unseen elements (microscopes, cross sections, chemical analysis, etc.)
Replicating the failure at will
The root cause analysis skill may rely on tools like x-rays and thermal imaging tools, some operated by specialists, yet you need to know which tools to employ and how to interpret their results. It may be fun to explore failures in a well furnished failure analysis lab, yet you need to focus on solving the mystery of what caused the failure.
You also need to be well versed in how to proceed from the “crime scene” (or instance of failure location), through symptoms, to non-destructive and destructive testing. You need to build your “case” based on evidence and logic, plus a healthy dose of engineering knowledge of the fundamental elements involved.
If working as the lone reliability engineer, you certainly need to establish an ongoing relationship with a failure analysis lab. In other words, do not rely on your vendors, do the failure analysis work under your organizations control with your own lab or contracted facility.
Get the information your team needs to solve problems or to avoid future problems by exercising your technical root causes analysis skills.
Good Reliability Work
To be good, I’m suggesting you have to have robust skills in statistics and root cause analysis. Do you agree? What else would you argue is essential to be a good reliability engineer?
Formal education (masters or Ph.D) or design/manufacturing engineering experience?
Where do you look when hiring a new reliability engineer? Do you head to U of Maryland or other university reliability program to recruit the top talent? Or, do you promote/assign from within? Where do yo find the best reliability people? Continue reading What makes the best Reliability Engineer?→
Not our personal or moral standards, rather the set of documents we rely upon as a foundation for reliability engineering tools and techniques.
We have a wide array of standards for reporting reliability test data to calculating confidence intervals on field returns. We have standards that describe various environmental conditions and appropriate testing levels suitable to evaluate your product. We define terms, concepts, processes, and techniques.
A Missing Element
Despite the many documents and impressive titles of numbers and abbreviations or acronyms, most of the standard related to reliability engineer fail to include sufficient context and rationale concerning when and why to use or modify the standard. If a specific test is to determine the expected lifetime of solder joints, well, which type of solder joints (shape, size, configuration, material, and process) is the standard appropriate and when does it not apply? Make the boundaries of applicability clear.
No single test works for all situations.
For example, a wrist watch standard defining how to test for specific water resistance claims does not evaluate the effects of corrosion. The standard has the watch or similar device exposed to a set of water conditions, then evaluate if the system is operating, nearly immediately after the water exposure.
We know that water encourages corrosion, yet takes time to occur. Water alone on a circuit board is no big deal (much of the time) it’s when the water facilitates the creation of additional and unwanted current paths that there is a problem. Metal migration and rusting, take time to occur.
If the standard for water resistance doesn’t evaluate corrosion, and it’s one of the ways your product fails, too bad. You can ‘pass’ the test, meet the standard, add it to your data sheet, and the customer will still experience a failure.
Same for many environmental testing, FMEA, life testing, field data analysis, and a range of other standards. They do not include the critical information necessary for appropriate application of the standard to your particular situation.
Connection to Value
Many, not all, standards provide a recipe to accomplish as task or evaluation. One of the values of the standard is different teams may replicate the results of one team by repeating the steps outlined in the standard.
One of issues with standards is they do not include how and why to actually accomplish the set of tasks and what to do with the results. In part, we need to clearly connect, say the task of testing a product across a range of temperature and humidity conditions, only if it will provide meaningful information.
Don’t run the test if the information is not needed, unnecessary or meaningless.
For example, if we expect that exposure to high temperature and humid conditions may increase the chance of product failure. We may want to know
how many failures will occur;
how the product will actually fail;
how the failure will initiate and progress;
when the failures occur under use conditions;
Or any number of reasons to use the results of the testing. Often we run a standard test with very few samples, experience no failures and erroneously conclude all it good. Then surprised that failures occur anyway when the product is in use.
The standard let us down.
The standard provided only a recipe or outline for a procedure and now that guidance and rationale on how it may or may not help us and our team resolve very real questions. Testing 3 units that all pass does not mean your solar panel will survive hot and humid conditions for 20 years with no failures. It doesn’t.
Only run the test or work to accomplish a process only if it is tied to answering a question. Focus on business decisions and the questions we have to resolve in order to make better decisions (i.e. Wrong less often).
Let’s change the way we read and use standards. You may need to add the how and why, the boundaries, and the connection to value for your situation. It’s not always easy. The people writing the standard often have sufficient experience to include guidelines to help you — when possible contact them and ask what was their thinking and what are the limitations.
If enough of us avoid simply meeting the requirements of the standard, we will
Enjoy reliable product performance
Create value to our organization with each test or task
One of the major dilemmas of reliability engineering is one we really need to solve. Too many times we are trapped by our organizations competing priorities and working with inadequate information.
We generally understand that field failure data provides the best possible representation of our product’s reliability performance. It’s data from our population of products with our customers while they apply all the stresses’ customer will apply to our product. Customer’s report the failures they care about, and not failures of little significance. Continue reading Finding the Hidden Field Data in Your Organization→
One of the twitter notes I sent out a few weeks ago in part read, “Celebrate failures”. And a comment came back that it was a wonderful approach that she had not though of before. Failure will occur and when it does it is our chance to learn.
And, we need to learn. As reliability professionals, we continue to learn our entire career. New materials fail in novel manners. New assemblies fail in an assortment of ways. New designs fail due to unknown sources of variation. We will see failures. So rather than simply focus on the next try and hope to find success, let’s learn from each failure as we move toward success. Continue reading Predicting Failure vs. Reacting to Failure→