AMC 20-152A: A practitioners perspective

A previous post discussed the high level impact of the new AMC 20-152A guidance on DO-254 programs. That post touched on elemental analysis and the new guidance which indicates DAL A and DAL B designs must collect and track more than just statement coverage. I did receive some feedback from practitioners requesting a deeper technical dive into the impact. Therefore, this post will discuss three fundamental activities in the quest to satisfy AMC 20-152 code coverage guidance.

Introduction

Code coverage comes in a variety of flavors and the new AMC 20-152A directs practitioners to collect and close on more than just statement coverage. This includes:

• Condition Coverage
• Line Coverage
• Branch Coverage
• and more.

AMC 20-152A states the following:

For hardware DAL A or DAL B, where HDL code coverage is used to perform elemental analysis (ED-80/DO-254 Appendix B Section 3.3.1), the applicant should define in the planning documents the detailed coverage criteria of the HDL code elements used in the design. The criteria should ensure coverage over the various cases of the HDL code elements used in the design (e.g. branches, conditions, etc.). Any non-covered case or element should be analyzed and justified.

Therefore, practitioners must now ensure their test harness adequately stimulates the design across each coverage scenario or provide the rationale for each coverage hole.

Code Coverage Recap

For those who are new to code coverage, below is a quick overview of the types of code coverage:

Statement Coverage

Indicates which statements in source code have been evaluated. This includes multiple statements on a single line or a single statement split across multiple lines.

Toggle Coverage

Indicates the instances a register or wire has toggled. Most commonly, verification engineers want to minimally see a zero-to-one and one-to-zero transition.

Branch Coverage

Indicates whether each branch condition has occurred in control structures present in the HDL. Control structures include if, case, while, repeat, and more.

Expression Coverage

Indicates whether a boolean operand has evaluated to both true and false. Note that expression coverage is sometimes referred to as “condition” coverage.

State Machine

Finite State Machine (FSM) coverage evaluates the number of times each state is entered and that each transition arc was exercised.

If you’d like to dive deep into the differences in the coverage types listed above, please visit the Verification Academy Coverage tutorial.

Automated coverage collection using Questa

Mentor’s Questa Simulator monitors and collects coverage during the execution of functional simulations. It’s as simple as adding a single switch to the optimization and simulation stages.

Optimization

vopt +cover[=<spec>] ...
    <spec> is used to enable code coverage metrics by type
    <spec> consists of one or more of the following letter codes:
         s (statement)
         b (branch)
         c (condition)
         e (expression)
         f (finite state machine)
         t (toggle)
         x (extended toggle

Example:  vopt +cover=sbfct

Simulation

Simply add the following to collect coverage during simulation

vsim -coverage …

In a perfect world, 100% code coverage is achieved after running the first regression. In reality, there will be coverage gaps during in the early stages of testing which will require debugging.

Debugging coverage gaps

The road to code coverage closure is an incremental one. Gaps in code coverage can be due to insufficient stimulus, or dead code due to HDL configuration or a bug. Siemens EDA offers formal solutions to identify unreachable code and deliver reference waveforms demonstrating how to hit existing coverage holes. Questa CoverCheck reads code coverage results from simulation via the UCDB and automatically (and exhaustively) analyses all the coverage holes. Additionally, because specifying waivers to deliberately exclude unused IP configurations from the analysis is a tedious and error prone manual process, an integrated waiver flow simplifies creating, maintaining, and documenting the rationale behind accepted coverage gaps.

If you are interested in hearing about how a practioner at Raytheon leveraged Formal at this years User2User, please see “Achieving 100% Code Coverage: What Used to Be Hard Is Now Easy With an Automated, Exhaustive Flow.”

Alongside formal, Siemens EDA offers next generation UI capabilities, enabling users to quickly evaluate coverage gaps, identify hot spots, and perform root cause and trend analysis.

By leveraging these two solutions, engineers are able to efficiently close coverage gaps, changing a traditionally unbounded timeline into a bounded one.

Establishing Traceability

Simply collecting code and functional coverage isn’t sufficient from an audit standpoint. Traceability must also be established to various design artifacts and DO-254 documentation. This process can be done manually or automated leveraging native Siemens EDA integrations. Below is a screen capture demonstrating automated traceability between Siemens Polarion implementing lifecycle management functionality and the underlying Siemens EDA tools.

Conclusion

If your program collected solely statement coverage in the past, the new AMC 20-152A guidance will impact verification teams as the effort to satisfy elemental analysis is greater. Toogle, Branch, and FSM are just a few of the new flavors of code coverage which must be closed in DO-254 programs. Siemens EDA offers solutions tailored to DO-254, which automate the collection of code coverage, debugging of coverage gaps, and traceability to documentation and design artifacts.