This past week I’ve heard from a few that have taken the ASQ Certified Reliability Engineer exam. The common comment was on the use of MTBF and the exponential distribution. It seems to emphasized.
That is bad.
MTBF, as those that read this blog regularly, know my opinion of the use of MTBF and the associated assumptions. In short, they lead to bad decisions due to an oversimplification of representations of the data.
The body of knowledge does mention many other distributions and not only exponential. Yet, it seems it is easier to write questions for the exponential or when using MTBF than others. Or the filtering of questions does not take into account the weight MTBF is being given in the exams. Or, those that craft the exams feel that MTBF is of value and should be well known and well tested by those testing for the CRE.
What ever the case – it should stop. Stop using, encouraging, testing and preparing for the use of MTBF. Reduce it’s importance and reduce it visibility. Maybe then our profession will start to learn and use on regular basis those tools and techniques that are truly useful. If one is faced with time to failure data, yes it is easier to calculate MTBF rather than a Weibull fit. Yet, we should be using Weibull as a minimum and the appropriate approach when justified.
We should not reinforce the notion that we can simply assume constant failure rates and drive on. In my opinion we need to break this cycle and do it now.
So if you are writing question – don’t use MTBF or exponential. If you’re crafting the exams – minimize the use of MTBF and exponential. And, if you’re taking the exam – provide feedback about the overemphasis of MTBF which isn’t appropriate as a reflection of the BoK.
Together we move this a bit and help the entire industry.
Fred.
Another great post.
I would like to add that the field of reliability engineering and in particular electronics reliability must change to emphasize deterministic methodologies to develop and improve reliability of our rapidly changing technologies. We must have many more engineers start to look at the real, not presumed, causes of field unreliability and seek to find and eliminate those causes before shipment to provide a value to electronics companies.
Kirk
Fred,
In practicality, what it takes to get a governing body or sponsoring body to modify a BOK and certification exam is monstrous, and the costs are dramatic. It costs a fortune to change the question set, even by a few questions, for any certification exam. Changing the BOK to support it would also be required, and that is a monster of a problem also. Imagine eliminating all references in the BOK that focus on MTBF. Imagine the outrage of the many university professors whose books that focus on MTBF if their books were not included anymore.
I’m not defending the CRE exam and BOK, but from my experience with INCOSE’s CSEP BOK and exam, even if you were able to clone your mindset into everybody on the CRE exam committee, it would take years to affect this change, many years. Part of the problem is that such a change would essentially invalidate certifications obtained under the current exams. A cheap and easy recertification process would have to be invented.
Then you have the huge investments made by companies buying tools based on MTBF that influence the CRE exam questions, as well as the tool vendors with tools based on MTBF. The amount of money at stake in all this that is affected by the CRE exam and training for it that it is incredible. The toolvendors would go beserk. (In fact, I would not be surprised if a number of tool vendors have you marked as someone to destroy professionally. I myself have been targeted by a few on LinkedIn.)
And, MTBF is not the only problem in the BOK and CRE exam. Many of the stats concepts used in both are just plain incorrect, e.g., 90% confidence intervals containing 90% probability, etc. I started commenting on some of your posts providing CRE exam help, pointing out where the exam has some of these things wrong, but stopped since passing the test is the point of your posts. As you know, most folks who take the CRE exam for certification are primarily checking a box for their job.
A dramatic and earth shattering change to correct the BOK and CRE exam would produce amazing improvements worldwide in RAM practice. Some companies would go broke, and new ones would get started and thrive, especially new training companies to pass the new MTBF-free and proper stats concept CRE exam . And the amount of money saved worldwide in RAM practice that would result from making less cost effective and sometime outright bad decisions based on MTBF would buy some small countries.
What you have recommended is about the best we can hope for. It will be a long time coming however.
Mark Powell
Fred,
Great post as always. You are asking for the world as Mark points out but you won’t get it unless you ask.
Mark,
I am somewhat offended by your comment about taking the CRE and checking a job box. Not really.
I have never worked at a company where being certified was a requirement but if I were able to make the CRE (or something comparable) a requirement for anyone practicing reliability, I would. Being certified is only peer recognition. I consider some sort of certification/degree a minimum requirement. A first step, if you will, in learning the discipline. Not the last. So I consider something like the CRE as a minimum requirement.
As we learn, we should change the BoK and that is a good thing. I don’t believe we all need to go back and get recertified just because the BoK has changed. Medical students a mere generation or so ago were taught about the benefits of smoking cigarettes and tobacco use in general. I have not heard of any medical degrees/licenses being revoked because we now know better. Closer to home, the half life of engineering knowledge was considered to be about 5 years. And that was when I was in school decades ago. The half life is much shorter now. If we were to force engineers to go back to school because of the advances made in engineering, we would all be in school and never be able to leave.
Just as my engineering degree was the first step in becoming an engineer, I consider the CRE as the first step in becoming a reliability engineer. I am all for changing the BoK, relegating MTBF to a historical footnote, and learning the proper reliability techniques.
Thanks to both of you for your insights.
Mark
Hi Mark F,
thanks for the support and kind words.
cheers,
Fred
Hi Mark,
Thanks for the note of encouragement (I think). I agree it will take years to make the change away from MTBF, yet if we don’t start, we never get there.
It’s like eating an elephant – one bite at a time.
cheers,
Fred
PS (never had nor intend to have elephant, yet the MTBF course is set).
Mark F.,
Sorry for the offense. I have worked on too many government contracts and proposals for government contracts that have had certification requirements (certain percentage of personnel on the contract had to have certain certifications). I have never actually met anyone who got any certification (CRE, ASP, CSP, CSEP, ESEP, PMP, etc.) for just “peer recognition.” With the expense and maintenance generally required of certifications, everybody I know who has gone for a cert of any type was to either qualify to work on a particular contract, to assure that the company could be competitive for certain types of contracts, or to be eligible for a promotion (that is what I meant by “checking a box”). I know that INCOSE (CSEP, ASEP certs) lobbies hard for the government to issue RFP’s with requirements for CSEP’s. That way they can get more folks to apply for the cert, and pay to take the test. Plus, the INCOSE cert exam training vendors make more money also. Certification is intended to be a profit center for professional societies, not to help the members. If you haven’t been in the inner circles of a professional society, involved with decisions on certification, that is not obvious at all.
Sorry again for the offense, none was intended. But I have never met anyone who got their cert just for “peer recognition.” Certs are awfully expensive, and expensive to maintain.
And with regard to an interest in recertification if the BOK and exam change dramatically, companies will indeed tout that they have personnel with the latest certs to distinguish them from folks with earlier certs based on the previous BOK and exam. This happens with all certs in a competitive environment, and it is a challenge that the professional societies that manage them (and enjoy the revenue that is generated) must face when considering BOK and exam changes. Which is why I said to Fred that even if everybody on the cert committee were to become zealous noMTBF advocates, it will still be a long time before any serious changes could be implemented.
Sorry again for the offense, none was intended.
Mark Powell
Mark,
As I mentioned, I was not offended. So no apology necessary.
Here is how ASQ defines their certification in general. I took this quote from their website moments ago. This is where I get the “peer recognition” phrase.
“Certification is formal recognition by ASQ that an individual has proficiency within, and a comprehension of, a specified body of knowledge. It is peer recognition, not registration or licensure”.
So it is peer recognition of a minimal level of comprehension and proficiency. What I called a first step. The required 3 year recertification is the method used to keep, at least minimally, current with the evolution of the field.
I am vaguely familiar with some of the requirements you mention for government projects. I was involved in writing parts of proposals etc. at times but not nearly as intimately involved as you have been. I remember requirements to list the qualifications of personnel on the contract so there may well have been requirements for a certain percentage to be degreed/certified. So, you are correct with the check the box comment.
The comment about certifications meant to be a profit center and not to help members is something I didn’t like reading but is far too often true. I have long felt this to be true in auditing circles with the old QS9000 and TS blah blah. It is the same with certifications as well. Depressing. Sigh…. So much work to do in our own house.
Looks like we will be eating elephant for a long time.
Mark
Mark,
Your quote from ASQ looks just like analogous quotes from every cert org I have read. No surprise as well, it is a selling quote.
I have worked a number of proposals where the RFP required a certain percentage of the proposed staff have specific certifications, and that that percentage had to be maintained for the duration of the contract. Such RFP’s and contracts are not uncommon at all. And, I know of contracts that were one by companies having more certified folks being proposed, even when not an RFP requirement.
Yeah, it is kind of a bummer to think that certification is for the professional society’s coffers rather than for the members and industries they are supposed to serve. Seen it directly in person for INCOSE and PMI, and they got the idea from other certifying professional societies. Certification is more of a racket than anything, literally if you think about it.
A word of advice, be wary of getting involved in top leadership in a professional society. The sausage never tastes quite as good if you can even stomach it after you have seen it being made.
Mark Powell