Is Engineering Best Judgement really “Best” ? (Part 1)

by Chuck Battikha, Solutions Architect for Verification & Functional Safety, Siemens EDA Services

To support the ISO 26262 standard, an automotive design must go through a safety analysis.

This is often done using an FMEDA (Failure Modes Effect and Diagnostic Analysis). This is a complex spreadsheet that attempts to determine several key metrics required by ISO 26262, notably:

Single Point Fault Metric (SPFM), a measure of how well the safety critical logic is protected by safety mechanisms.

Latent Fault Metric (LFM), a measure of how well the safety mechanisms are protected.

What this FMEDA needs to capture (among many things) is a list of the elements within a design (modules, blocks, etc.) and how well those elements would be protected by safety mechanisms, measured by Diagnostic Coverage (DC):

List of Elements	% of overall Design	Failure Rate of Element	Effective DC	Main Safety Mechanism	Residual Failure Rate	Etc.
Block A	10%	10% of BFR	90%	CRC	(100%-90%) x  10% x BFR)
Block B	5%	5% of BFR	60%	Parity	(100%-90%) x  (5% x BFR)
Etc.
Total	100%	–	–	–	Sum of Column

Figure 1: Representative FMEDA

The SPFM and LFM are equations generated at the top level and based on totals.

SPFM = 1 – (Total of Residual Failure Rate / Total Failure Rate for Safety Related Elements).

Safety Mechanisms can range from redundancy to parity to ECC to CRC and implemented in either hardware, software or both. Some of the material in the FMEDA is mechanical and straightforward- an example of this is listing the blocks and their normalized contribution by gates, area, or transistor count.

Using your “Judgement”…

However, the derivation of some of the information needed can be considered an art form. The “How to” of finding the DC of your safety mechanisms can be addressed using Engineering Best Judgement. The ISO 26262 standard (in both Part 5 Appendix D and Part 11, Clause 5) assigns High, Medium, and Low as labels for coverage at 99%, 90%, and 60%, respectively. The standard even gives recommendations of DC in these sections that engineers can use as justifications.

Pretty straightforward, eh?

However, there are some challenges:

1. Part 5 Appendix D states that the tables “are not exhaustive and other techniques can be used, provided evidence is available to support the claimed diagnostic coverage.”
2. The standard includes phrases like “effectiveness depends...”, “…can reduce diagnostic coverage“, and “higher coverage is possible…”.

So in the end, for engineers using Engineering Best Judgement in filling out their FMEDA, there’s quite a bit that can be left to interpretation (or imagination). This leeway, then, leads to a few questions – among them are:

• What are the ramifications of these interpretations?
• Why is there an interpretation?
• Is there a better way?

As for ramifications, there are two competing trade-offs. If your FMEDA determines that a block is better covered by a safety mechanism (or set of mechanisms) than it really is, the metrics (SPFM and LFM) will be higher than they should be – in other words, the design may not be safe enough. An incorrect FMEDA may, then be caught by a customer or an auditor, which can then result in a loss of sales, loss of certification, and almost certainly a pricey reworking of work products or even of the design itself.

Not good.

Worse, an incorrect FMEDA may not be caught… until the SoC/IP has been designed into an automobile and the error found after a post-crash investigation.

Also, not good.

Now, if you are too conservative, your design can be needlessly over-engineered, resulting in higher cost due to the increased area, effort and development time. This can result in missed market windows and lack of sales due to uncompetitive pricing. Duplicating all the logic in a design, for example, would certainly make it safer- but at (at least) twice the cost.

In the end, you need confidence that your FMEDA is correct.

FMEDA: Are all created equally?

Now why is there a range of interpretations in a design? In short, it is due to the complexity of that design’s elements and, in a typical SoC, the large number of blocks to analyze. The work can be overwhelming due to both the amount of detail that is required and to the subjective nature of the detailed analysis. Years of experience have shown it highly unlikely that independent engineers creating FMEDAs for the same design will consistently get the same answer, though they might agree on many areas (e.g. calling out High coverage for two blocks that are in lockstep, or for a memory protected by ECC).

But, what happens in a more complex example like a custom processing block with a combination of timers, periodic and post diagnostics, parity on data paths, and possibly other safety mechanisms intermixed throughout?

Almost assuredly, there would be differences in rationals and level of details based on each engineer’s understanding of the design specifics, level of experience, time allocated to the task, schedule pressures, etc.

So…

How would one FMEDA be better than the other?

How would we know?

Even though both FMEDAs are based on Engineering Best Judgements, we are still left to wonder:

Which is really Best?

——————————————————————————————————————-

Look for more to come in Part 2 of our examination of the FMEDA… coming soon to this blog page !

In the meantime, you can learn more about easing the FMEDA process from Siemens EDA experts in this free whitepaper: PUSH-BUTTON FMEDAS FOR AUTOMOTIVE SAFETY AUTOMATING A TEDIOUS TASK

If you’ve questions about FMEDA, Functional Safety or how Siemens EDA Services can help you maximize your verification success, we invite you to contact us at: SiemensEDAServices.sisw@siemens.com.