{"id":9064,"date":"2013-07-15T04:48:09","date_gmt":"2013-07-15T11:48:09","guid":{"rendered":"https:\/\/blogs.mentor.com\/verificationhorizons\/?p=9064"},"modified":"2026-03-27T08:34:50","modified_gmt":"2026-03-27T12:34:50","slug":"part-5-the-2012-wilson-research-group-functional-verification-study","status":"publish","type":"post","link":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/2013\/07\/15\/part-5-the-2012-wilson-research-group-functional-verification-study\/","title":{"rendered":"Part 5: The 2012 Wilson Research Group Functional Verification Study"},"content":{"rendered":"

<\/strong>\u00a0<\/p>\n

Effort Spent in Verification<\/strong><\/p>\n

This blog is a continuation of a series of blogs that present the highlights from the 2012 Wilson Research Group Functional Verification Study (click here<\/a>). In my previous blog (click here<\/a>), I focused on design and verification reuse trends. In this blog, I focus on the controversial topic of the amount of effort spent in verification.<\/p>\n

Directly asking study participants how much effort they spend in verification will not work. The reason is that it\u2019s hard to find a paper or article on verification that doesn\u2019t start with the phrase: \u201cSeventy percent of a project\u2019s effort is spent in verification\u2026\u201d In other words, the industry is already biased to respond with this effort value. Yet, there are really no creditable references to quantify this value.<\/p>\n

I don\u2019t believe that there is a simple answer to the question, \u201cHow much effort was spent on verification in your last project?\u201d In fact, I believe that it is necessary to look at multiple data points derived from multiple questions to truly get a sense of effort spent in verification. And that\u2019s what we did in our functional verification study.<\/p>\n

Total Project Time Spent in Verification<\/strong><\/p>\n

To try to assess the effort spent in verification, let\u2019s begin by looking at one data point, which is the total project time spent in verification. Figure 1 shows the trends in total percentage of project time spent in verification for non-FPGA designs by comparing the 2007 Far West Research study (in gray), the 2010 Wilson Research Group study (in blue), and the 2012 Wilson Research Group study (in green).\u00a0<\/p>\n

\"\"<\/a><\/p>\n

Figure 1. Percentage of total project time spent in verification for Non-FPGA designs<\/strong><\/p>\n

The graph clearly shows that there are some projects that spend a significant percentage of project time in verification (>80%), while other projects spend significantly less time. Notice that in 2007, the average (mean) project time spent in verification was 49 percent, while the average increased to 56 percent in 2010 and remained the same in 2012.<\/p>\n

Figure 2 shows the trends in total percentage of project time spent in verification for FPGA designs by comparing the 2010 Wilson Research Group study (in pink) and the 2012 Wilson Research Group study (in red).<\/p>\n

\"\"<\/a>\u00a0<\/p>\n

Figure 2. Percentage of total project time spent in verification for FPGA designs<\/strong><\/p>\n

You might note that many FPGA projects tend to spend less time in verification than non-FPGA projects. Traditionally, the strategy for FPGA designs has been to get to the lab as soon as possible and debug issues in the lab. In a future blog I\u2019ll show data that indicates this strategy does not necessarily yield good results in terms of meeting project schedule or quality objectives.<\/p>\n

Peak Number of Design and Verification Engineers<\/strong><\/p>\n

Next, let\u2019s look at another data point, the average (mean) peak number of engineers involved on a project. Figure 4 compares the growth in recent years for the average peak number of design engineers (in light green) and verification engineers (in dark green) working on a typical non-FPGA project.<\/p>\n

\u00a0\"\"<\/a><\/p>\n

Figure 3. Peak number of design vs. verification engineer trends for non-FPGA projects<\/strong><\/p>\n

Note that there has not been a significant increase in design engineers in the past five years, although design sizes have continued to increase at a Moore\u2019s Law rate. This is partially due to increased adoption of internal and external IP (as I discussed in my previous blog) as well as continued productivity improvements due to automation.<\/p>\n

However, the mean peak number of verification engineers working on non-FPGA projects has increased by 75% within the last five years. In fact, today we see (on average) a one-to-one ratio for a project\u2019s peak number of design and verification engineers.<\/p>\n

Figure 4 provides a different analysis of the data by partitioning the projects by design sizes, and then calculating the mean peak number of verification engineers by project design. The design size partitions are represented as: less than 5M gates, 5M to 20M gates, and greater than 20M gates.<\/p>\n

\"\"<\/a>\u00a0<\/p>\n

Figure 4. Mean peak number of verification engineer trends by design size for non-FPGA projects<\/strong><\/p>\n

Figure 5 shows the average (mean) peak number of design engineers (in red) and verification engineers (in pink) working on a typical FPGA project.<\/p>\n

\u00a0\"\"<\/a><\/p>\n

Figure 5. Peak number of design vs. verification engineer trends for non-FPGA projects<\/strong>\u00a0<\/p>\n

Also, note that the ratio of design engineers versus verification engineers hasn\u2019t changed within the last two years for FPGA projects. Typically, design engineers on FPGA projects are responsible for verification too, and you will find many projects that do not have verification engineers. This trend, however,\u00a0will likely change\u00a0as FPGA designs become more complex. We are already seeing this on some very complex FPGA projects today.<\/p>\n

In my next blog (click here<\/a>), I\u2019ll continue the discussion on effort spent in verification as revealed by the 2012 Wilson Research Group Functional Verification Study.<\/p>\n","protected":false},"excerpt":{"rendered":"

\u00a0 Effort Spent in Verification This blog is a continuation of a series of blogs that present the highlights from…<\/p>\n","protected":false},"author":71592,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spanish_translation":"","french_translation":"","german_translation":"","italian_translation":"","polish_translation":"","japanese_translation":"","chinese_translation":"","footnotes":""},"categories":[1],"tags":[326,506,718,751,819,820,827],"industry":[],"product":[],"coauthors":[],"class_list":["post-9064","post","type-post","status-publish","format-standard","hentry","category-news","tag-accellera","tag-functional-verification","tag-simulation","tag-systemverilog","tag-verification","tag-verification-academy","tag-verification-methodology"],"_links":{"self":[{"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/posts\/9064","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/users\/71592"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/comments?post=9064"}],"version-history":[{"count":1,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/posts\/9064\/revisions"}],"predecessor-version":[{"id":19753,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/posts\/9064\/revisions\/19753"}],"wp:attachment":[{"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/media?parent=9064"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/categories?post=9064"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/tags?post=9064"},{"taxonomy":"industry","embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/industry?post=9064"},{"taxonomy":"product","embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/product?post=9064"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/coauthors?post=9064"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}