Verification Methodology Reset
Discussion around verification methodologies have been going on for a couple decades. It started back around 2000 with the emergence of the various hardware verification languages (HVLs). Verification specific languages lead to the divergence of design and verification and the creation of the verification engineer. It was HVLs that enabled the transition of directed testing toward constrained random. Early in constrained random adoption it became obvious supporting infrastructure was necessary to be productive; early frameworks included RVM, eRM, AVM, VMM and various homegrown alternatives. The first step in consolidating an approach to constrained random happened with OVM; the final step was UVM which is where we are now.
As time passed and habits formed, UVM became the thing most of us identify with when someone says the word methodology. Constrained random is assumed to be part of that equation being that it’s the workhorse for most teams in our industry. Reuse, also closely associated with methodology, is the primary driver of technical integration with emerging techniques like portable stimulus; UVM, of course, helps form the foundation for that technical integration.
Even if you think beyond UVM, verification methodology radiates from a verification point of view. While born of exploding design complexity, methodologies live solely within verification teams.
Verification_methodology.reset().
I propose reevaluating our current definition of verification methodology. I’d like to see renewed focus on design being the central deliverable and verification methodologies conceived and driven by the needs of design. I’d like to see us break a design into a series of abstractions, identify options for verifying each abstraction, employ techniques best suited to each abstraction and integrate those techniques so transitions are made effectively.
That’s a mouthful. Agree or disagree I think it makes for worthy discussion.
In breaking a design into a set of abstractions, here are five that I identify with: line, block, feature, feature set and system.
A line of code is self explanatory; that’s as low level as it gets. Code block is next; also self explanatory though important to note that code blocks are localized within a design unit (I almost said module there but changed it so I didn’t lose anyone!). Features are where we cross the threshold into characteristics that users identify with yet still don’t provide much value on their own. Simple features will still be localized but more complex features will have a distributed implementation that crosses design unit boundaries. Next are feature sets; definitely distributed across subsystems and the first indication of an actual deliverable. Our highest design abstraction is a complete system where integrated feature sets delivery real-world value.
Examples from each abstraction to help you identify…
- Line: write enable asserted
- Block: write enable pipeline
- Feature: store/load to/from L1 cache
- Feature set: cache coherence
- System: Multicore SoC w/memory subsystem
Identifying options for each abstraction is the fun part of this exercise now that I work for a tool provider. Like I said in Building Integrated Verification Flow – Round 2, I’ve done this before focusing only on simulation techniques due to my own experience being limited to simulation. With Mentor it’s totally different. Suddenly I work with experts that have used all the tools for years and they’re more than happy to share their experience. When you get a cross section of experts sharing their experience, this is what you get…
This is a superset of techniques that Mentor provides or enables. Needless to say, there is a lot here (One of my marketing colleagues described it as busy which I’m ok with because it gets the point across that there are a lot of options to consider).
You see I have each technique plotted as design abstraction v. verification abstraction. We already talked about design abstraction; verification abstraction is the level at which we describe interactions in our tests. For example, X-Check is a tool that aims at low level design characteristics (i.e. single line assignments) with properties applied at a bit level; constrained random testing is meant for randomly exercising design features with tests written at a transaction level. And so it goes.
Summarizing why I’ve captured it this way: design abstraction v. test abstraction shows us the sweet spot for a collection of techniques. We see which are best suited to each design abstraction while understanding the test abstraction where they are most effectively applied. The suggestion here is to use tools and techniques within their sweet spot; anywhere else and the value they provide tapers off.
But what’s in all these bubbles? And what do the colours mean?
Good questions. In our next post, I’ll go through each with an explanation and some reasoning for why they are where they are.
-neil
PS: Another reminder that this is work in progress. If you see something you like, something I’m missing, something you disagree with or anything else, please let me know in the comments or send me a note at: neil_johnson@mentor.com.
