{"id":13307,"date":"2018-11-14T10:07:08","date_gmt":"2018-11-14T17:07:08","guid":{"rendered":"https:\/\/blogs.mentor.com\/verificationhorizons\/?p=13307"},"modified":"2026-03-27T08:39:04","modified_gmt":"2026-03-27T12:39:04","slug":"prologue-the-2018-wilson-research-group-functional-verification-study","status":"publish","type":"post","link":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/2018\/11\/14\/prologue-the-2018-wilson-research-group-functional-verification-study\/","title":{"rendered":"Prologue: The 2018 Wilson Research Group Functional Verification Study"},"content":{"rendered":"<p>This is the first in a sequence of blogs that presents the findings from our new 2018 Wilson Research Group Functional Verification Study. Similar to my previous 2016 Wilson Research Group functional verification study blogs, I plan to begin this sequence of blogs with an exclusive focus on FPGA trends. Why? For the following reasons:<\/p>\n<ul>\n<li>Some of the more interesting trends in our 2018 study are related to FPGA designs.<\/li>\n<li>The FPGA market has a difficult time with non-trivial bug escapes into production.<\/li>\n<li>The FPGA market is rapidly maturing its verification processes to address growing complexity.<\/li>\n<li>The IC\/ASIC market has converged on common processes driving by maturing industry standards.<\/li>\n<li>The IC\/ASIC market is fairly mature in its adoption of various verification technology and techniques for IP and subsystem verification. Many of the new IC\/ASIC challenges have moved to the system level.<\/li>\n<\/ul>\n<h2>Study Overview<\/h2>\n<p>The study results presented in this paper are a continuation of a series of industry studies on functional verification. This series includes the previously published 2014 and 2016 Wilson Research Group Functional Verification Study [3][4]. Each of these studies was modeled after the 2002 and 2004 Collett International Research, Inc. studies [1][2] and focus on the IC\/ASIC market. While we began studying the FPGA market in 2012, we waited until we had sufficient multi-year data points to identify verification trends to draw any significant conclusions.<\/p>\n<p>For the purpose of our study, the sampling frame was constructed from eight industry lists that we acquired. This enabled us to cover all regions of the world and all relevant electronics industry market segments. It is important to note that we did not include our own account team\u2019s customer list in the sampling frame. This was done in a deliberate attempt to prevent vendor bias in the final results. While we architected the study in terms of questions and then compiled and analyzed the final results, we commissioned Wilson Research Group to execute our study. After data cleaning the results to remove inconsistent, incomplete, or random responses, the final sample size consisted of 1205 eligible and qualified participants (i.e., n=1205).<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-13308\" src=\"https:\/\/blogs.sw.siemens.com\/wp-content\/uploads\/sites\/54\/2018\/11\/2018-0-1.png\" alt=\"\" width=\"1280\" height=\"720\" \/><\/p>\n<p><strong>Figure 1: Study participants by targeted implementation<\/strong><\/p>\n<p>Figure 1 compares the percentage of 2016 and 2018 study participants (i.e., design projects) by targeted implementation for both IC\/ASIC and FPGA projects. It is important to note that targeted implementation does not represent silicon volume in terms of the global semiconductor market since a single project could account for a significant portion of semiconductor market revenue. However, the data suggest that projects creating designs targeted at high-performance SoC programmable FPGAs is increasing, which is one indication of growing FPGA complexity.<\/p>\n<h2>Confidence Interval<\/h2>\n<p>Since all survey-based studies are subject to sampling errors, we attempt to quantify this error in probabilistic terms by calculating a confidence interval. For our study, we determined the overall margin of error to be \u00b14% using a 95% confidence interval. In other words, this confidence interval tells us that if we were to take repeated samples from a population, 95% of the samples would fall inside our margin of error \u00b14%, and only 5% of the samples would fall outside.<\/p>\n<h2>Study Participants<\/h2>\n<p>This section provides background on the makeup of the study.<\/p>\n<p>The 2018 Wilson Research Group studies were worldwide studies. The regions targeted were:<\/p>\n<ul>\n<li>North America<\/li>\n<li>Europe<\/li>\n<li>Mideast<\/li>\n<li>Africa<\/li>\n<li>Pacific Rim\/Asia<\/li>\n<li>India<\/li>\n<\/ul>\n<p>The survey results are compiled both globally and regionally for analysis. In this blog sequence we will present the global trends.<\/p>\n<p>Figure 2 shows the percentage of overall study FPGA and IC\/ASIC participants by market segment. It is important to note that this figures does not represent silicon volume by market segment.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-13314\" src=\"https:\/\/blogs.sw.siemens.com\/wp-content\/uploads\/sites\/54\/2018\/11\/2018-0-2-1.png\" alt=\"\" width=\"1280\" height=\"720\" \/><\/p>\n<p>Figure 2: FPGA and IC\/ASIC study participants by market segment<\/p>\n<p>Figure 3 shows the percentage of overall study eligible FPGA and IC\/ASIC participants by their job description. An example of eligible participant would be a self-identified design or verification engineer, or engineering manager, who is actively working within the electronics industry. Overall, design and verification engineers accounted for a majority of the study participants.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-13315\" src=\"https:\/\/blogs.sw.siemens.com\/wp-content\/uploads\/sites\/54\/2018\/11\/2018-0-3-1.png\" alt=\"\" width=\"1280\" height=\"720\" \/><\/p>\n<p>Figure 3: FPGA and IC\/ASIC study participants\u2019 job title description<\/p>\n<p>Before I start presenting the findings from our 2018 functional verification study, I plan to discuss in my next blog (<a href=\"https:\/\/blogs.mentor.com\/verificationhorizons\/blog\/2018\/11\/15\/understanding-and-minimizing-study-bias-2018-study\/\" target=\"_blank\" rel=\"noopener\">click here<\/a>) general bias concerns associated with all survey-based studies\u2014and what we did to minimize these concerns.<\/p>\n<h2>Quick links to the 2018 Wilson Research Group Study results<\/h2>\n<ul>\n<li><a href=\"https:\/\/blogs.mentor.com\/verificationhorizons\/blog\/2018\/11\/14\/prologue-the-2018-wilson-research-group-functional-verification-study\/\" target=\"_blank\" rel=\"noopener\">Prologue: The 2018 Wilson Research Group Functional Verification Study<\/a><\/li>\n<li><a href=\"https:\/\/blogs.mentor.com\/verificationhorizons\/blog\/2018\/11\/15\/understanding-and-minimizing-study-bias-2018-study\/\" target=\"_blank\" rel=\"noopener\">Understanding and Minimizing Study Bias (2018 Study)<\/a><\/li>\n<li><a href=\"https:\/\/blogs.mentor.com\/verificationhorizons\/blog\/2018\/11\/19\/part-1-the-2018-wilson-research-group-functional-verification-study\/\" target=\"_blank\" rel=\"noopener\">Part 1 \u2013 FPGA Design Trends<\/a><\/li>\n<li><a href=\"https:\/\/blogs.mentor.com\/verificationhorizons\/blog\/2018\/11\/26\/part-2-the-2016-wilson-research-group-functional-verification-study-2\/\" target=\"_blank\" rel=\"noopener\">Part 2 \u2013 FPGA Verification Effectiveness Trends<\/a><\/li>\n<li><a href=\"https:\/\/blogs.mentor.com\/verificationhorizons\/blog\/2018\/12\/04\/part-3-the-2018-wilson-research-group-functional-verification-study\/\" target=\"_blank\" rel=\"noopener\">Part 3 \u2013 FPGA Verification Effort Trends<\/a><\/li>\n<li><a href=\"https:\/\/blogs.mentor.com\/verificationhorizons\/blog\/2019\/01\/02\/part-4-the-2018-wilson-research-group-functional-verification-study\/\" target=\"_blank\" rel=\"noopener\">Part 4 \u2013 FPGA Verification Effort Trends (Continued)<\/a><\/li>\n<li><a href=\"https:\/\/blogs.mentor.com\/verificationhorizons\/blog\/2019\/01\/07\/part-5-the-2018-wilson-research-group-functional-verification-study\/\" target=\"_blank\" rel=\"noopener\">Part 5 \u2013 FPGA Verification Technology Adoption Trends<\/a><\/li>\n<li><a href=\"https:\/\/blogs.mentor.com\/verificationhorizons\/blog\/2019\/01\/15\/part-6-the-2018-wilson-research-group-functional-verification-study\/\" target=\"_blank\" rel=\"noopener\">Part 6 \u2013 FPGA Verification Language and Library Adoption Trends<\/a><\/li>\n<li><a href=\"https:\/\/blogs.mentor.com\/verificationhorizons\/blog\/2019\/01\/22\/part-7-the-2018-wilson-research-group-functional-verification-study\/\" target=\"_blank\" rel=\"noopener\">Part 7 \u2013 IC\/ASIC Design Trends<\/a><\/li>\n<li><a href=\"https:\/\/blogs.mentor.com\/verificationhorizons\/blog\/2019\/01\/29\/part-8-the-2018-wilson-research-group-functional-verification-study\/\" target=\"_blank\" rel=\"noopener\">Part 8 \u2013 IC\/ASIC Resource Trends<\/a><\/li>\n<li><a href=\"https:\/\/blogs.mentor.com\/verificationhorizons\/blog\/2019\/02\/05\/part-9-the-2018-wilson-research-group-functional-verification-study\/\" target=\"_blank\" rel=\"noopener\">Part 9 \u2013 IC\/ASIC Verification Technology Adoption Trends<\/a><\/li>\n<li><a href=\"https:\/\/blogs.mentor.com\/verificationhorizons\/blog\/2019\/02\/14\/part-10-the-2018-wilson-research-group-functional-verification-study\/\" target=\"_blank\" rel=\"noopener\">Part 10 \u2013 IC\/ASIC Language and Library Adoption Trends<\/a><\/li>\n<li><a href=\"https:\/\/blogs.mentor.com\/verificationhorizons\/blog\/2019\/03\/05\/part-11-the-2018-wilson-research-group-functional-verification-study\/\" target=\"_blank\" rel=\"noopener\">Part 11 \u2013 IC\/ASIC Power Management Trends<\/a><\/li>\n<li><a href=\"https:\/\/blogs.mentor.com\/verificationhorizons\/blog\/2019\/03\/13\/part-12-the-2018-wilson-research-group-functional-verification-study\/\" target=\"_blank\" rel=\"noopener\">Part 12 \u2013 IC\/ASIC Verification Results Trends<\/a><\/li>\n<li><a href=\"https:\/\/blogs.mentor.com\/verificationhorizons\/blog\/2019\/03\/19\/conclusion-the-2018-wilson-research-group-functional-verification-study\/\" target=\"_blank\" rel=\"noopener\">Conclusion: The 2018 Wilson Research Group Functional<\/a><\/li>\n<\/ul>\n<h2>References<\/h2>\n<p>[1]\u00a0\u00a0\u00a0\u00a0 R. Collett, \u201c2002 IC\/ASIC functional verification study,\u201d Industry Report from Collett International Research, Inc. 2003.<\/p>\n<p>[2]\u00a0\u00a0\u00a0\u00a0 R. Collett, \u201c2004 IC\/ASIC functional verification study,\u201d Industry Report from Collett International Research, Inc. 2005.<\/p>\n<p>[3]\u00a0\u00a0\u00a0\u00a0 H. Foster, Trends in functional verification: a 2014 industry study, Proceedings of the 52nd Annual Design Automation Conference (DAC), June 07-11, 2015, San Francisco, California.<\/p>\n<p>[4]\u00a0\u00a0\u00a0\u00a0 H. Foster, Trends in functional verification: a 2016 industry study, DVCon 2017, San Jose, California.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>This is the first in a sequence of blogs that presents the findings from our new 2018 Wilson Research Group&#8230;<\/p>\n","protected":false},"author":71592,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spanish_translation":"","french_translation":"","german_translation":"","italian_translation":"","polish_translation":"","japanese_translation":"","chinese_translation":"","footnotes":""},"categories":[1],"tags":[326,350,401,419,493,504,506,528,533,535,577,623,638,718,751,785,787,819,820,827,831,833],"industry":[],"product":[],"coauthors":[],"class_list":["post-13307","post","type-post","status-publish","format-standard","hentry","category-news","tag-accellera","tag-assertion-based-verification","tag-coverage","tag-debug","tag-formal-verification","tag-functional-coverage","tag-functional-verification","tag-ieee","tag-ieee-1800","tag-ieee-1801","tag-low-power","tag-ovm","tag-portable-stimulus","tag-simulation","tag-systemverilog","tag-upf","tag-uvm","tag-verification","tag-verification-academy","tag-verification-methodology","tag-verilog","tag-vhdl"],"_links":{"self":[{"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/posts\/13307","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/users\/71592"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/comments?post=13307"}],"version-history":[{"count":1,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/posts\/13307\/revisions"}],"predecessor-version":[{"id":19866,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/posts\/13307\/revisions\/19866"}],"wp:attachment":[{"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/media?parent=13307"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/categories?post=13307"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/tags?post=13307"},{"taxonomy":"industry","embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/industry?post=13307"},{"taxonomy":"product","embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/product?post=13307"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/coauthors?post=13307"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}