{"id":11481,"date":"2015-04-21T10:27:34","date_gmt":"2015-04-21T17:27:34","guid":{"rendered":"https:\/\/blogs.mentor.com\/verificationhorizons\/?p=11481"},"modified":"2026-03-27T08:36:03","modified_gmt":"2026-03-27T12:36:03","slug":"part-4-the-2014-wilson-research-group-functional-verification-study","status":"publish","type":"post","link":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/2015\/04\/21\/part-4-the-2014-wilson-research-group-functional-verification-study\/","title":{"rendered":"Part 4: The 2014 Wilson Research Group Functional Verification Study"},"content":{"rendered":"

FPGA Verification Effectiveness Trends<\/strong><\/h2>\n

This blog is a continuation of a series of blogs related to the 2014 Wilson Research Group Functional Verification Study (click here<\/a>). \u00a0In my previous blog (click here<\/a>), I focused on the amount of effort spent in FPGA verification. We have seen in previous blogs that a significant amount of effort is being applied to FPGA functional verification. In this blog I focus on the effectiveness of verification in terms of FPGA project schedule and bug escapes.<\/p>\n

FPGA Schedules<\/strong><\/h3>\n

Figure 1 presents the design completion time compared to the project\u2019s original schedule. What was a surprise in the 2014 findings is that we saw an improvement in the number of FPGA projects meeting schedule\u2014compared to 2012. It is unclear why we are seeing this trend now.\u00a0 Perhaps managers are getting better at scheduling\u2014or are becoming more pessimistic with their schedules.\u00a0 Or, perhaps it is due to the increase amount of reuse (both design and verification IP). Or, is the increased amount of FPGA verification effort prior to “getting to the lab” starting to pay off for some projects? This data point raises some interesting questions worth exploring further. Regardless, still a significant number of FPGA projects miss their originally planned schedule.<\/p>\n

\"2014-WRG-BLOG-FPGA-4-1\"<\/a><\/p>\n

Figure 1. FPGA design completion time compared to the project\u2019s original schedule<\/strong><\/p>\n

<\/h3>\n

FPGA Lab Iterations<\/strong><\/h3>\n

ASIC\/IC projects track the number of required spins that occur prior to market production.\u00a0 In fact, this can be a useful metric for determining the overall verification effectiveness of an ASIC\/IC project.\u00a0 Unfortunately, we lack such a metric for FPGA projects.\u00a0 For the 2014 study, we decided to ask the question related to the average number of lab iterations required before the design went into production. Again, this was done to try and get a sense of the project\u2019s verification effectiveness.\u00a0 The results are shown in Figure 2. However, I’m not convinced that FPGA lab iterations is analogous to ASIC\/IC respin as a verification effectiveness metric.\u00a0 Perhaps a better metric for future studies would be the number of bugs that escape into production and are found in the field. This might be something we should consider on future studies.<\/p>\n

\"2014-WRG-BLOG-FPGA-4-2\"<\/a><\/p>\n

Figure 2. Number of FPGA iterations in the lab (no trend data available)<\/strong><\/p>\n

<\/h3>\n

FPGA Bug classification<\/strong><\/h3>\n

For the 2014 study, we asked the FPGA project participants to identify the type of flaws that were contributing to rework in the lab. In Figure 3, I show the two leading causes of rework, which are logical and functional bugs, as well as clocking bugs. The data seems to suggest that these issues are growing. Perhaps due to the design of larger and more complex FPGAs. Again, this is a data point worth exploring further.<\/p>\n

\"2014-WRG-BLOG-FPGA-4-3\"<\/a><\/p>\n

Figure 3. Types of Flaws Resulting in FPGA Rework<\/strong><\/p>\n

In Figure 4, I show trends in terms of main contributing factors leading to logic\u00a0and functional flaws\u2014and you can see that design errors are the main cause of functional flaws.\u00a0 But note that a significant amount of flaws are related to some aspect of the specification\u2014such as changes in the specification\u2014or incorrect or incomplete specifications. Problems associated with the specification process are a common theme I often hear when visiting FPGA customers.<\/p>\n

\"2014-WRG-BLOG-FPGA-4-4\"<\/a><\/p>\n

Figure 5. Root cause of FPGA functional flaws<\/strong><\/p>\n

In my next blog (click here), I plan to presenting the findings from our study for FPGA verification technology adoption trends.<\/p>\n

Quick links to the 2014 Wilson Research Group Study results<\/h2>\n