{"id":4711,"date":"2011-06-26T20:09:50","date_gmt":"2011-06-27T03:09:50","guid":{"rendered":"https:\/\/blogs.mentor.com\/verificationhorizons\/?p=4711"},"modified":"2026-03-27T08:34:14","modified_gmt":"2026-03-27T12:34:14","slug":"part-9-the-2010-wilson-research-group-functional-verification-study","status":"publish","type":"post","link":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/2011\/06\/26\/part-9-the-2010-wilson-research-group-functional-verification-study\/","title":{"rendered":"Part 9: The 2010 Wilson Research Group Functional Verification Study"},"content":{"rendered":"

<\/a>Verification Techniques & Technologies Adoption Trends<\/span><\/strong><\/h3>\n

This blog is a continuation of a series of blogs, which present the highlights from the 2010 Wilson Research Group Functional Verification Study (<\/span>for a background on the study, click here<\/span><\/a>). <\/span><\/p>\n

In my previous blog (<\/span>Part 8 click here<\/span><\/a>), I focused on some of the 2010 Wilson Research Group findings related to design and verification language trends. In this blog, I present verification techniques and technologies adoption trends, as identified by the 2010 Wilson Research Group study. <\/span><\/p>\n

One of the claims I made in the prologue to this series of blogs is that we are seeing a trend in increased industry adoption of advanced functional verification techniques, which is supported by the data I present in this blog. An interesting question you might ask is “what is driving this trend?”\u00a0In some\u00a0of my earlier blogs (click here for <\/span>Part 1<\/span><\/a> and <\/span>Part 2<\/span><\/a>) I showed an industry trend in that design complexity is increasing in terms design sizes and number of embedded processors. In addition, I\u2019ve presented trend data that showed an increase in total project time and effort spent in verification (click here for <\/span>Part 4<\/span><\/a>). My belief is that the industry is being forced to mature its functional verification process to address increasing complexity and effort.<\/span><\/p>\n

Simulation Techniques Adoption Trends<\/span><\/strong><\/h3>\n

Let\u2019s begin by comparing \u00a0the adoption trends related to various simulation techniques as shown in Figure 1, where the data from the 2007 Far West Research study\u00a0 is shown in blue and the data from 2010 Wilson Research Group study is shown in green.<\/span>\u00a0<\/span>\u00a0<\/span>\"\"<\/a><\/p>\n

Figure 1. Simulation-based technique adoption trends<\/span><\/strong><\/p>\n

You can see that the study finds the industry increasing its adoption of various functional verification techniques.<\/span><\/p>\n

For example, in 2007, the Far West Research Group found that only 48 percent of the industry performed code coverage. This surprised me. After all, HDL-based code coverage is a technology that has been around since the early 1990\u2019s. However, I did informally verify the 2007 results through numerous customer visits and discussions. In 2010, we see that the industry adoption of code coverage has increased to 72 percent. <\/span><\/p>\n

Now, a side comment: In this blog, I do not plan to discuss either the strengths or weaknesses of the various verification techniques that were studied (such as code coverage, whose strengths and weaknesses have been argued and debated for years)\u2014perhaps in a separate series of future blogs. In this series of blogs, I plan to focus only on the findings from the 2010 Wilson Research Group study. <\/span><\/p>\n

In 2007, the Far West Research Group study found that 37 percent of the industry had adopted assertions for use in simulation. In 2010, we find that industry adoption of assertions had increased to 72 percent. I believe that the maturing of the various assertion language standards has contributed to this increased adoption.<\/span><\/p>\n

In 2007, the Far West Research Group study found that 40 percent of the industry had adopted functional coverage for use in simulation. In 2010, the industry adoption of functional coverage had increased to 69 percent. Part of this increase in functional coverage adoption has been driven by the increased adoption of constrained-random simulation, since you really can\u2019t effectively do constrained-random simulation without doing functional coverage.<\/span><\/p>\n

In fact, we see from the Far West Research Group 2007 study that 41 percent of the industry had adopted constrained-random simulation techniques. In 2010, the industry adoption had increased to 69 percent. I believe that this increase in constrained-random adoption has been driven by the increased adoption of the various base-class library methodologies, as I presented in a previous blog (click here for <\/span>Part 8<\/span><\/a>).<\/span><\/p>\n

Formal Property Checking Adoption Trends<\/strong><\/span><\/h3>\n

Figure 2 shows the trends in terms of formal property checking adoption by comparing the 2007 Far West Research study (in blue) with the 2010 Wilson Research Group study (in green). The industry adoption of formal property checking has increased by an amazing 53 percent in the past three years. Again, this is another data point that supports my claim that the industry is starting to mature its adoption of advanced functional verification techniques.<\/span><\/p>\n

\"\"\u00a0<\/span><\/p>\n

Figure 2. Trends in formal<\/a>\u00a0property checking adoption<\/span><\/strong><\/p>\n

Another way to analyze the results is to partition a project\u2019s adoption of formal property checking by design size, as shown in Figure 3, where less than 1M gates\u00a0 is shown in blue, 1M to 20M gates is shown in orange, and greater than 20M gates is shown in red. Obviously, the larger the design, the more effort is generally spent in verification. Hence, it\u2019s not too surprising to see the increased adoption of formal property checking for larger designs.<\/span><\/p>\n

\"\"<\/a><\/span><\/p>\n

Figure 3. Trends in formal property checking adoption by design size<\/span><\/strong><\/p>\n

Acceleration\/Emulation & FPGA Prototyping Adoption Trends<\/span><\/strong><\/h3>\n

The amount of time spent in a simulation regression is an increasing concern for many projects. Intuitively, we tend to think that the design size influences simulation performance. However, there are two equally important factors that must be considered: number of test in the simulation regression suite, and the length of each test in terms of clock cycles.\u00a0 <\/span><\/p>\n

For example, a project might have a small or moderate-sized design, yet verification of this designs requires a long running test (e.g., a video input stream). Hence, in this example, the simulation regression time is influenced by the number of clock cycles required for the test, and not necessarily the design size itself.<\/span><\/p>\n

In <\/span>blog 6<\/span><\/a> of this series, I presented industry data on the number of tests created to verify a design in simulation (i.e., the regression suite). The findings obviously varied dramatically from a handful of test to thousands of test in a regression suite, depending on the design. In Figure 4 below, I show the findings for a projects regression time, which also varies dramatically from short regression times for some projects, to multiple days for other projects. The median simulation regression time is about 16 hours in 2010.<\/span>\u00a0\"\"<\/a><\/span><\/p>\n

Figure 4. Simulation regression time trends<\/span><\/strong><\/p>\n

One technique that is often used to speed up simulation regression (either due to very long test and lots of test)\u00a0is either hardware assisted acceleration or emulation. In addition, FPGA prototyping, while historically used as a platform for software development, has recently served a role in SoC integration validation.<\/span><\/p>\n

Figure 5 shows the adoption trend for both HW assisted acceleration\/emulation and FPGA prototyping by comparing the 2007 Far West Research study (in blue) with the 2010 Wilson Research Group study (in green). We have seen a 75 percent increase in the adoption of HW assisted acceleration\/emulation over the past three years.<\/span>\u00a0\"\"<\/a><\/span><\/p>\n

Figure 5. HW assisted acceleration\/emulation and FPGA Prototyping trends<\/span><\/strong><\/p>\n

I was surprised to see that the adoption of FPGA prototyping did not increase over the past three years, considering that we found in increase in SoC development over the same period. So, I decided to dig deeper into this anomaly. <\/span><\/p>\n

In Figure 6, you\u2019ll see that I partitioned the HW assisted acceleration\/emulation and FPGA prototyping adoption data by design size: less than 1M gates (in blue), 1M to 20M gates (in yellow), and greater than 20M gates (in red). This revealed that the adoption of HW assisted acceleration\/emulation continued to increase as design sizes increased. However, the adoption of FPGA prototyping rapidly dropped off as design sizes increased beyond 20M gates.\u00a0\u00a0<\/span>\u00a0\"\"<\/a><\/span><\/p>\n

Figure 6. Acceleration\/emulation & FPGA Prototyping adoption by design size <\/span><\/strong><\/p>\n

So, what\u2019s going on? One\u00a0problem with FPGA prototyping of large designs is that there is an increased engineering effort required to partition designs across multiple FPGAs. In fact, what I have found is that FPGA prototyping of very large designs is often a major engineering effort in itself, and that many projects are seeking alternative solutions to address this problem.<\/span><\/p>\n

In my next blog, I plan to present the 2010 Wilson Research Group study findings related to various project results in terms of schedule and required spins before production.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

Verification Techniques & Technologies Adoption Trends This blog is a continuation of a series of blogs, which present the highlights…<\/p>\n","protected":false},"author":71592,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spanish_translation":"","french_translation":"","german_translation":"","italian_translation":"","polish_translation":"","japanese_translation":"","chinese_translation":"","footnotes":""},"categories":[1],"tags":[457,493,506,718,758],"industry":[],"product":[],"coauthors":[],"class_list":["post-4711","post","type-post","status-publish","format-standard","hentry","category-news","tag-emulation","tag-formal-verification","tag-functional-verification","tag-simulation","tag-testbench"],"_links":{"self":[{"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/posts\/4711","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/users\/71592"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/comments?post=4711"}],"version-history":[{"count":1,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/posts\/4711\/revisions"}],"predecessor-version":[{"id":19737,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/posts\/4711\/revisions\/19737"}],"wp:attachment":[{"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/media?parent=4711"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/categories?post=4711"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/tags?post=4711"},{"taxonomy":"industry","embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/industry?post=4711"},{"taxonomy":"product","embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/product?post=4711"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/verificationhorizons\/wp-json\/wp\/v2\/coauthors?post=4711"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}