{"id":13064,"date":"2018-05-03T10:58:40","date_gmt":"2018-05-03T17:58:40","guid":{"rendered":"https:\/\/blogs.mentor.com\/verificationhorizons\/?p=13064"},"modified":"2026-03-27T08:38:49","modified_gmt":"2026-03-27T12:38:49","slug":"verification-is-from-vulcan-validation-is-from-pandora","status":"publish","type":"post","link":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/2018\/05\/03\/verification-is-from-vulcan-validation-is-from-pandora\/","title":{"rendered":"Verification is from Vulcan, Validation is from Pandora"},"content":{"rendered":"

At DVCon earlier this year, I was lucky enough to present to the munching masses at the Wednesday lunch. Now, some folks treat the DVCon\u00a0lunch as a captive audience, and serve up some stodgy fare indeed.
\nThat\u2019s a pity.
\nAs an industry, we are a highly intelligent,\u00a0knowledgeable and discerning crowd;\u00a0and it\u2019s a mystery to me why some folks should think that all that switches off\u00a0when we sit down to lunch. Do we really\u00a0become sponges to soak up any sales pitch dumped on us, or do we just want to have lunch and some down time?<\/p>\n

So anyway, the Mentor approach is to provide an oasis\u00a0from the serious business of the proper DVCon programme, and I think folks are grateful for that. Please, tell me if I\u2019m wrong.<\/p>\n

However, that doesn\u2019t mean that we don\u2019t have some serious thoughts and questions to raise over dessert. This year, we compared and contrasted the different disciplines of Verification and Validation. We plan to repeat a couple of the arguments in Blogs over the coming weeks.<\/p>\n

So, what’s all this about Vulcan and\u00a0Pandora?<\/strong><\/p>\n

In many ways, Verification is a finite logical problem i.e. \u201cwhat is the chance that my hardware has bugs?\u201d and modern coverage-driven techniques can give us a logical metric-based answer to that question, with the aim of getting as close to \u201cno chance\u201d as possible. Our agony in Verification is that we NEVER reach \u201cno chance\u201d so we are left to decide when we are \u201cclose enough\u201d, and the trade-off between conscience and pragmatism begins.<\/p>\n

Validation, on the other hand, is not so objective and logical. Before a design starts, decisions have already been made about its desired form, function, space and pace. Not just in terms of the hard measurable parameters such as performance and power, but also \u201csofter\u201d factors, such as aesthetics, user interface, environmental impact or market acceptance criteria (long live focus groups!).<\/p>\n

The ideal validation environment might involve building a series of progressively better versions of the design and exposing each to an infinite number of monkeys until one is sure that it is perfectly fit for purpose, and nothing can be broken; accidentally or deliberately. That\u2019s the ideal but in the same way that the \u201cno chance\u201d Verification ideal is never reached, neither is complete Validation. This is partly owing to cost and time, those two omnipresent barriers to perfection in any project, but also largely because some aspects of Validation success are measured against subjective criteria, not logical ones (hence the Vulcan and Pandora references, geddit?).<\/p>\n

I made the analogy of buying a shirt in order to highlight the difference between Verification and Validation. This is summarised in the picture below, but if you’d like the explanation to go with it, may I humbly refer you to Brian Bailey’s review of Mentor’s DVCon lunch presentation<\/a><\/p>\n

 <\/p>\n

\"\"<\/a><\/p>\n

 <\/p>\n

It\u2019s the software, stupid<\/strong><\/p>\n

A lot of the distinction between Verification and Validation tasks comes down to the design\u2019s software content. Ah yes, but which software? Consider a software stack, with the lowest levels such as the boot code or the Board Support Package being most closely dependent upon the hardware functionality; and the upper levels of the stack i.e. the application and user space, being completely divorced from it. Typically, Verification requires just enough of the software stack in order to exercise the hardware under test; whereas Validation needs all of it \u2013 the full chip \u2013 the whole stack.<\/p>\n

Some engines such as emulation and simulation can indeed accurately run software, but not all of it owing to a lack of execution speed. Even FPGA prototypes, the fastest pre-silicon engine, may lack that required speed for full stack operation. Often hybrid techniques are used; exiling some of the System functionality into a transaction level approximation, such as an Arm\u00ae Fast Model. In this way, an emulator can run enough of the software to do some serious hardware-software co-validation.<\/p>\n

Taken to an extreme, the whole system can be modelled at the transaction level, creating a virtual prototype, which is not just a pre-silicon but also a pre-RTL approximation as well; in fact an excellent environment in which to validate user interfaces and aesthetics.<\/p>\n

Some Really Useful Engines<\/strong><\/p>\n

These illustrations show that, while each engine represents a different trade-off between speed and accuracy, no engine should be considered solely as a Verification engine or a Validation engine; each could be either . . , or both!<\/p>\n

So, it seems that we have a wide range of pre-silicon engines at our disposal, so what\u2019s the problem?<\/p>\n

The problem is that we can\u2019t simply switch to the best engine when we want; it simply takes too much effort and time to bring-up the design in any particular engine, or hybrid of those engines, so teams may often continue to use an engine beyond its optimal Field of Use for their particular design. How do we make that engine change-over simpler?<\/p>\n

Having a common \u201cfront-end\u201d and common Verification IP (surely that should be Verification and Validation IP \u2013 Ed.) is a good goal towards which some EDA vendors are proprietorially progressing but today there is still a wide gulf in expertise and re-use between engines; a gulf that needs to be crossed.<\/p>\n

That\u2019s another pity.<\/p>\n

Just imagine running an accurate full-chip, whole-stack validation environment, on an FPGA prototype, say, and then at the proverbial push-of-a-button, switching viewpoints and zooming in on a mixed-signal simulation of some key cells. That kind of viewpoint change would allow us to throw a searchlight on typical validation tasks; for example investigating the fluctuations of a specific voltage rail while the software stack is running multiple applications using real-world data. Surely that is desirable. But, it is the difficulty in switching viewpoints \u2014 to switch accuracy-speed trade-off \u2014 that is a limitation in today\u2019s Validation environments.<\/p>\n

Keep it Safe, Keep it Secret<\/strong><\/p>\n

Then there\u2019s Safety and Security \u2013 probably the strongest drivers of Validation today. Validation might be considered as testing the readiness of your design for exposure to the rigors of real life. Some (in)famous validation failures in the past took many years and weird corner-case analysis to be uncovered (the Meltdown scenario is a case in point) however, most others became apparent rather too quickly (hmmm . . . my battery seems to be getting rather warm).<\/p>\n

It\u2019s your design vs the rest of the world; be careful out there. Eventually, all systems have to deal with the messy, analogue world, where other unpredictable and possibly malicious systems (and people) may break even the most heavily verified design i.e. Verification alone is not enough. The more complete and accurate one\u2019s Validation scenarios, the less likely one\u2019s design is going to be floored during the first round.<\/p>\n

Ding-ding!<\/p>\n

 <\/p>\n

Doug – May 4th, 2018<\/p>\n","protected":false},"excerpt":{"rendered":"

At DVCon earlier this year, I was lucky enough to present to the munching masses at the Wednesday lunch. Now,…<\/p>\n","protected":false},"author":71587,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spanish_translation":"","french_translation":"","german_translation":"","italian_translation":"","polish_translation":"","japanese_translation":"","chinese_translation":"","footnotes":""},"categories":[1],"tags":[442,811],"industry":[],"product":[],"coauthors":[],"class_list":["post-13064","post","type-post","status-publish","format-standard","hentry","category-news","tag-dvcon","tag-validation"],"_links":{"self":[{"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/posts\/13064","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/users\/71587"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/comments?post=13064"}],"version-history":[{"count":1,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/posts\/13064\/revisions"}],"predecessor-version":[{"id":19859,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/posts\/13064\/revisions\/19859"}],"wp:attachment":[{"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/media?parent=13064"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/categories?post=13064"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/tags?post=13064"},{"taxonomy":"industry","embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/industry?post=13064"},{"taxonomy":"product","embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/product?post=13064"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/coauthors?post=13064"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}