{"id":13373,"date":"2018-11-26T13:28:17","date_gmt":"2018-11-26T20:28:17","guid":{"rendered":"https:\/\/blogs.mentor.com\/verificationhorizons\/?p=13373"},"modified":"2026-03-27T08:39:09","modified_gmt":"2026-03-27T12:39:09","slug":"part-2-the-2016-wilson-research-group-functional-verification-study-2","status":"publish","type":"post","link":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/2018\/11\/26\/part-2-the-2016-wilson-research-group-functional-verification-study-2\/","title":{"rendered":"Part 2: The 2018 Wilson Research Group Functional Verification Study"},"content":{"rendered":"

This blog is a continuation of a series of blogs related to the 2018 Wilson Research Group Functional Verification Study (click here<\/a>).\u00a0 In my previous blog (click here<\/a>), I focused on FPGA design trends. In this blog, I present the findings from our new study related how successful FPGA projects are in terms of verification effectiveness.<\/p>\n

FPGA Verification Effectiveness<\/h2>\n

Non-Trivial Bug Escapes<\/h3>\n

IC\/ASIC projects have often used the metric \u201cnumber of required spins before production\u201d as a benchmark to assess a project\u2019s verification effectiveness. Historically, about 30% of IC\/ASIC projects are able to achieve first silicon success, and most successful designs are productized on the second silicon spin. Unfortunately, FPGA projects have no equivalent metric. As an alternative to IC\/ASIC spins, our study asked the FPGA participants \u201chow many non-trivial bugs escaped into production?\u201d The results shown in Fig. 2-1 are somewhat disturbing. In 2018, only 16% of all FPGA projects were able to achieve no bug escapes into production, which is worse than IC\/ASIC in terms of first silicon success, and for some market segments, the cost of field repair can be significant. For example, in the mil-aero market, once a cover has been removed on a system to upgrade the FPGA, the entire system needs to be revalidated.<\/p>\n

\"\"<\/p>\n

Fig. 2-1. Non-trivial FPGA bug escapes into production<\/strong><\/p>\n

Types of Flaws Resulting in Non-Trivial Bug Escapes<\/h3>\n

Fig. 2-2 shows various categories of design flaws contributing to FPGA non-trivial bug escapes. While the data suggest an improvement in percentage of \u201clogic or functional flaws,\u201d it remains the leading cause of bugs. This reduction of \u201clogic and functional flaws\u201d is likely due to the FPGA market maturing its verification processes, which we will quantify in upcoming blogs as well as increased adoption of mature design IP for integration.<\/p>\n

\"\"<\/p>\n

\u00a0Fig. 2-2. Types of flaws resulting in FPGA bug escapes<\/strong><\/p>\n

Design Completion Compared to Original Schedule<\/h3>\n

In addition to bug escape metrics that we used to determine an FPGA project\u2019s effectiveness, another metric we tracked was project completion to the original schedule, as shown in Fig. 2.3. Here we found that 64% of FPGA projects were behind schedule. One indication of growing design and verification complexity is reflected in the increasing number of FPGA projects missing schedule during the period 2014 through 2018.<\/p>\n

\"\"<\/p>\n

Fig. 2-3. Actual FPGA project completion compared to original schedule<\/strong><\/p>\n

In my next blog (click here<\/a>), I’ll focus on verification effort trends related to FPGA designs.<\/p>\n

Quick links to the 2018 Wilson Research Group Study results<\/h2>\n