{"id":12666,"date":"2017-01-03T09:24:45","date_gmt":"2017-01-03T16:24:45","guid":{"rendered":"https:\/\/blogs.mentor.com\/verificationhorizons\/?p=12666"},"modified":"2026-03-27T08:38:04","modified_gmt":"2026-03-27T12:38:04","slug":"conclusion-the-2016-wilson-research-group-functional-verification-study","status":"publish","type":"post","link":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/2017\/01\/03\/conclusion-the-2016-wilson-research-group-functional-verification-study\/","title":{"rendered":"Conclusion: The 2016 Wilson Research Group Functional Verification Study"},"content":{"rendered":"

Deeper Dive into First Silicon Success and Safety Critical Designs<\/h2>\n

This blog is a continuation of a series of blogs related to the 2016 Wilson Research Group Functional Verification Study (click here<\/a>).\u00a0 In my previous blog (click here<\/a>), I presented verification results in terms of schedules, number of required spins, and classification of functional bugs. In this blog, I conclude the series by having a little fun with some of the findings from our\u00a0recent study.<\/p>\n

You might recall\u00a0from our 2014 study we did a deeper dive into the findings made a non-intuitive observation related to design size and first silicon success. That is, the smaller the design the less likelihood of achieving first silicon success (see 2014 conclusion blog<\/a> for details). This observation still\u00a0holds true in 2016.<\/p>\n

For our 2016 study, we decided to do a deeper dive related to the following:<\/p>\n

    \n
  1. Verification maturity and silicon success, and<\/li>\n
  2. Safety critical designs and silicon successes.<\/li>\n<\/ol>\n

    Verification Maturity and Silicon Success <\/strong><\/h3>\n

    Figure 1\u00a0presents the findings for\u00a0ASIC\/IC first silicon success, and Figure 2\u00a0presents the findings\u00a0for FPGA\u00a0non-trivial bug escapes into production. It is important to note that only 33 percent of ASIC\/IC projects are able to achieve first silicon success, and only 22 percent of FPGA projects go into production without a non-trivial bug.<\/p>\n

    \"BLOG-2016-WRG-figure-Con-1\"<\/p>\n

    Figure 1. ASIC\/IC required spins before final production<\/strong><\/p>\n

    \"BLOG-2016-WRG-figure-Con-2\"<\/p>\n

    Figure 2. FPGA non-trivial bug escapes into production<\/strong><\/p>\n

    A question worth asking is if there might be some\u00a0correlation between project success (in terms of preventing bugs) and verification maturity. To answer this question we looked at verification technology adoption trends related to a project’s silicon success.<\/p>\n

    Figure 3\u00a0presents the adoption of various verification techniques related to ASIC\/IC projects, and then correlates these results against achieving first silicon success. The data suggest that the more mature an ASIC\/IC project is in its adoption of verification technology the more likelihood of achieving first silicon success.<\/p>\n

    \"BLOG-2016-WRG-figure-Con-3\"<\/p>\n

    Figure 3. ASIC\/IC spins and verification maturity.<\/strong><\/p>\n

    Similarly, in Figure 4 we examine the adoption of various verification techniques on FPGA projects, and then correlate these results against preventing non-trivial bug escapes into production. Again, the data suggest that the more mature an FPGA project is in its adoption of verification technology the more likelihood that a non-trivial bug will not escape into production.<\/p>\n

    \"BLOG-2016-WRG-figure-Con-4\"<\/p>\n

    Figure 4. FPGA non-trivial bug escapes into production and verification maturity.<\/strong><\/p>\n

    Safety Critical Designs and Silicon Success<\/strong><\/h3>\n

    The second aspect of our 2016 study that we decided to examine a little deeper relates to safety critical designs and their silicon success. Intuitively, one might think that the rigid and structured process required to adhere to one of the safety critical development processes (such as, DO-254 for mil\/aero, ISO 26262 for automotive, IEC 60601 for medical, etc.) would yield higher quality in terms of preventing bugs and achieving silicon success.<\/p>\n

    Figure 5 shows the percentage of ASIC\/IC and FPGA projects that claimed to be working on a safety critical design.<\/p>\n

    \"BLOG-2016-WRG-figure-Con-5\"<\/p>\n

    Figure 5. Percentage of projects working on safety critical designs<\/strong><\/p>\n

    Keep in mind that the findings in Figure 5 do not represent volume in terms of silicon production\u2014the data\u00a0represents projects that claim to work under one of the safety critical development process standards.\u00a0 Using this partition between projects\u00a0working on\u00a0non-safety\u00a0critical and safety critical designs we decided to see how these two classes of projects compared in terms of preventing bugs.<\/p>\n

    Figure 6 compares the number of required spins for both safety critical and non-safety critical ASIC\/IC designs. While Figure 7 compares the FPGA designs with non-trivial bug escapes for both safety critical and non-safety critical designs.<\/p>\n

    \"BLOG-2016-WRG-figure-Con-6\"<\/p>\n

    Figure 6. Requires ASIC\/IC spins for safety critical vs. non-safety critical designs<\/strong><\/p>\n

    \u00a0\"BLOG-2016-WRG-figure-Con-7\"<\/strong><\/p>\n

    Figure 7. Non-trivial bug escapes for safety critical vs. non-safety critical FPGA designs<\/strong><\/p>\n

    Clearly, the data suggest that a development process adopted to ensure safety does not necessarily ensure quality. Perhaps this is non-intuitive. However, to be fair, many of the safety critical features implemented in today’s designs are quite complex and increase the verification burden.<\/p>\n

    This concludes the findings from the 2016 Wilson Research Group Study.<\/p>\n

    Quick links to the 2016 Wilson Research Group Study results<\/strong><\/p>\n