Decoding LLM Hallucinations: Insights and Taming them for EDA Applications
In an earlier blog, I explained that LLM will learn much faster than humans and there are many possible applications in verification. Two months later, the latest iteration of ChatGPT has been passing many harder exams with flying colors. This has ignited many people’s interest in applying LLMs in more domains including EDA. However, the immediate future may not be as rosy as we might have thought.
One of LLMs’ biggest weaknesses is hallucination, exemplified by two lawyers who copied hallucinated bogus cases directly from ChatGPT without fact-checking and a defamation suit over a non-existent embezzlement complaint generated by ChatGPT.
LLMs will still take time to replace EDA engineers. Hallucination and many intrinsic limitations will put them firmly into the assistant seat until a fundamental change in how LLMs work.
Universal Approximation Theorem and Its Implication
To better understand how an LLM works, we’ll have to trace back to the general ML’s working mechanism. In a rule-based system, the knowledge is learned by the domain experts, summarized into explicit and explainable rules, then coded by a programmer into an executable algorithm. This is in sharp contrast with data-centric systems. Like the supervised ML system in the figure, they exploit the intrinsic structure of the training data to implicitly learn the rules from the training data and automatically build the model to approximate the target function.
The Universal Approximation (UA) Theorem has proven that an Artificial Neural Network (ANN) can approximate the behavior of any continuous function in a certain range at arbitrary accuracy. The proof for the recurrent neural network (RNN) was given by our Siemens friends Anton Maximilian Schäfer et al. in 2006 [1]. The UA theorem has multiple implications:
- An ANN can only approximate a given function: Except for a few very special cases, most ANNs will only be Good Enough TM approximations of their respective target functions.
- It can only do so within a given input range: it is undefined what will happen to the out-of-range cases.
- The target function shall be continuous: In most cases, we don’t even know if the target function is continuous, and we may only obtain very limited samples from the problem space.
UA’s Implications to LLMs
Do the implications listed above apply to LLMs? The answer is also a definitive yes. The proof for the transformer (a special ANN architecture serving as the foundation of all LLMs) was given in a paper titled “Are Transformers universal approximators of sequence-to-sequence functions?” [2] What does it mean to apply LLMs to solve EDA problems?
An LLM’s answer can only be regarded as an approximation. Depending on the model performance, it shall not be regarded as the accurate or optimal answer. Especially when the employed training data might be intrinsically inaccurate, or less optimal like scraped or crowd-sourced data used to train these LLMs. It may work for some natural language problems, but such behaviors are not desired when precision and rigor are highly valued. Just as LLMs can still make grammatical mistakes, they tend to make more mistakes in solving EDA problems.
For any answer that is out-of-range of the training data of an LLM, the result will be unpredictable. In the cases where the training samples are dense, and the model is well trained, the results may be reasonable — but there is no guarantee. The story of the 5-color traffic controller design in Verilog in the last blog is just one of many examples some LLMs may give outlandish and unexpected answers to some out-of-range prompts.
Most real-world problem spaces are very likely discontinuous. Many ML models work well for their respective problems because the low quality answers can be tolerated or the problem spaces are near-continuous. However, this is not the case for natural languages or programming languages. Most natural languages are highly irregular and ambiguous. For natural languages, look at many forms of irregular inflections of English verbs. While programming languages may have well-defined simple grammar, their implementation complexity is still highly discontinuous. For example, we may initialize two variables with 2 assignment statements, but we rarely do it for 100 variables’ initialization.
Hallucination of LLMs in EDA
Analyzing the UA’s implications will help us better understand the restrictions of LLMs, and therefore helps better apply them in EDA.
Hallucination because data in EDA are sparse. As of mid-2022, our survey indicates there are about 10m lines of code and documents in 40 thousand HDL projects hosted on GitHub, RISC-V and OpenCores, regardless of their licenses. But it is easily dwarfed by the scale of other popular programming languages, like JavaScript, Java, or Python with 6.8m, 5.7m, and 3.5m projects respectively [3]. With sparse samples, these LLMs’ performance on HDL is expected to be less ideal than most popular programming languages. It can be remedied to a certain extent because many programming languages may share similar concepts learned from natural languages or other programming languages.
Hallucination due to (lack of ) data quality. Quality data are not easily accessible. Despite open-source data availability, most large-scale and highest-quality data are still confined in their individual silos. They probably will never be available to train a generally useable LLM. We are cautiously optimistic that the situation may improve over time when more and more quality designs are made available by the community and generous contributors. The rise of RISC-V may further catalyze and accelerate this openness process.
Hallucination when the task is out of range. Unprepared users may be surprised by LLMs pretentious answers about some non-existent people and their publications. It happens when the question is beyond the scope of datasets used for training. It can be remedied by grounding the answers to the actual data in the datasets. The trained model will give a higher rank to the answers with a real reference instead of the pretentious ones. The most recent update of the ChatGPT model tries to provide references with more real sources behind them if you ask. Many of these references may be less relevant to the questions, but at least they do exist.
Hallucination happens when an LLM doesn’t fit well. It was estimated that training of GPT-3 would cost millions of dollars. While it was never disclosed, training of GPT-4 would be in the tens of millions of dollars range. Training an LLM used to be so expensive that many LLMs are reported to be undertrained. The situation is rapidly changing with the help of newer-generation GPUs and specialized accelerators for LLMs. The arrival of finetuning techniques like LoRA has also made it possible to finetune a model at more affordable cost. The new models are expected to be better trained and fit the data better than their predecessors.
Outlook
Although hallucination is a new term coined for generative ML models, it shares many common traits with other ANN models. Due to the UA theorem implication, hallucination will never be entirely eliminated from LLMs if they’re still based on neural network models. We’ll have to live with hallucination and find creative ways to alleviate its undesired impacts on the applications.
Besides the countermeasures discussed above, the results generated by LLMs must still be carefully inspected and verified by their users and/or proven methodologies. Various control mechanisms must be imposed to maximize the productivity boost brought forward by LLMs to meet ever-higher precision and rigor requirements in EDA.
The data-driven verification methodology promoted by Questa verification products helps our users focus on gathering verification data from the entire verification cycle and extracting value from the data. Once many high-quality data are available, a well-trained LLM will prove to be more robust with less tendency to hallucinate.
If history teaches us anything, a revolutionary event’s short-term impacts tend to be overestimated, while its long-term impacts tend to be underestimated. It seems the invention of LLMs might be such an event. At Siemens, we still firmly believe in bringing value to our users, and LLMs will serve as invaluable assistants to the productivity boost. At the same time, their pitfalls will be carefully guarded by Siemens’ rigorous verification technologies.
[1] Anton Maximilian Schäfer, Hans Georg Zimmermann. “Recurrent neural networks are universal approximators.” Artificial Neural Networks–ICANN 2006: 16th International Conference, Athens, Greece, September 10-14, 2006. Proceedings, Part I 16. Springer Berlin Heidelberg, 2006.
[2] Chulhee Yun, Srinadh Bhojanapalli, Ankit Singh Rawat, Sashank J. Reddi, Sanjiv Kumar. “Are transformers universal approximators of sequence-to-sequence functions?.” arXiv preprint arXiv:1912.10077 (2019).
[3] Dan Yu, Harry Foster, Siemens EDA; Tom Fitzpatrick. “A Survey of Machine Learning Applications in Functional Verification.” DVCon 2023
Update Jun 19, 2023: My colleague Mark Hampton pointed out that I have an hallucination. As of now ChatGPT does not provide links to sources yet, unless you explicitly ask for them.
Comments
Leave a Reply
You must be logged in to post a comment.
View the whole ✳️ Examine every page of the Stop and Shop weekly advertisement for this week’s Stop and Shop Circular and next week’s Stop and Shop Flyer.
Get your coupons ready for the early ✳️ Plan your shopping trip ahead of time with the Stop & Shop Weekly Circular!
The Stop & Shop advertisement for this week is quite easy to read. Make sure you visit Stop & Shop often to take advantage of all the new weekly promotions.
https://stopandshoppweeklyad.store/
View the Fresh Thyme Weekly Ad for this week and the Fresh Thyme Ad for next week! Use the left and right arrows to view every page in the Fresh Thyme Weekly Flyer.
Plan your shopping trip ahead of time and get your coupons ready for the early Fresh Thyme weekly ad circular.
The advertisement for Fresh Thyme this week is quite easy to read. The sales at Fresh Thyme grocery store are broken down into various categories to make it easy to find out if the item you’re looking for is on sale or not. The next week, make sure to check the weekly circular to see if the item you want to buy is on sale.
https://freshthyme-weeklyad.shop/
View the full ✳️ Publix Weekly Ad for this week and the preview of the Publix Ad for next week! Examine each page of the Publix weekly advertisement circular.
The new advertisement starts on Wednesday or Thursday and runs for a week, depending on your store (check your business to see which schedule they utilize). Get your coupons ready for the early ✳️ and plan your shopping trip ahead of time. A sneak peek at the weekly Publix advertisement including some bogo deals!
You can save a substantial sum of money by looking over the Publix ads for next week and choosing the best deals! Don’t miss the amazing deals in the upcoming weekly Publix ad!
https://ipublixweeklyad.shop/
This week’s Food Lion Weekly Ad and next week’s Food Lion Ad! Go through every page of the Food Lion Weekly Circular. Prepare your coupons for the early Food Lion deals and schedule your shopping trip in advance!
This week’s Food Lion sales advertisement is quite simple to peruse. To make it simple to determine if the item you’re looking for at the Food Lion grocery shop is on sale or not, the sales are divided into different categories.
https://foodlion-weeklyad.info/
Next week is the Food City Ad, and this week is the Food City Weekly Ad! The Food City weekly ad circular’s pages can be navigated by using the left and right arrows. Get your coupons ready for the early Food City weekly ad preview and schedule your shopping trip in advance!
https://foodcityweeklyad.info/
This week’s and next week’s Whole Foods weekly advertisements! To move through every page of the Whole Foods Ad Flyer, use the left and right arrows.
Prepare your coupons for the early Whole Foods weekly ad circular and schedule your shopping trip in advance.
https://wholefodsweeklyad.info/
The Harris Teeter weekly flier is incredibly simple to peruse. This week’s Harris Teeter sales are divided into categories to make it simple to determine whether or not the product you’re looking for is on sale.
By browsing the Harris Teeter advertisement for the upcoming week and selecting the best offers, you can make significant savings!
https://harristeeterweeklyad.online/
This week’s Dollar General Weekly Ad and next week’s Dollar General Ad! Go through every page of the weekly sales advertisement for Dollar General. Prepare your coupons for the upcoming Dollar General Weekly Ad!
Look through the Dollar General ad preview to find the hottest offers and save a ton of money! Don’t pass up the fantastic deals in the weekly flyer from Dollar General!
https://dollargeneralweeklyad.info/
View the whole ✳️ Examine every page of the Stop and Shop weekly advertisement for this week’s Stop and Shop Circular and next week’s Stop and Shop Flyer.
Get your coupons ready for the early ✳️ Plan your shopping trip ahead of time with the Stop & Shop Weekly Circular!
Take advantage of the best deals in the Stop and Shop Circular for next week to save a ton of money! Don’t miss the amazing deals in the upcoming weekly advertisement from Stop and Shop!
stop and shop weekly circular print
It is quite simple to peruse the Smart and Final weekly flier. To make it simple to determine whether the product you’re looking for is on sale during this week’s Smart and Final sales, the sales are divided into different categories. Regardless of the item you choose to purchase,
https://smartandfinalweeklyad.us/
View the full ✳️ Publix Weekly Ad for this week and the preview of the Publix Ad for next week! Examine each page of the Publix weekly advertisement circular.
The new advertisement starts on Wednesday or Thursday and runs for a week, depending on your store (check your business to see which schedule they utilize). Get your coupons ready for the early ✳️ and plan your shopping trip ahead of time. A sneak peek at the weekly Publix advertisement including some bogo deals!
Looking through the Publix weekly flier is really easy. This week’s Publix sales are grouped to make it easy to find out if the product you’re looking for is on sale.
https://ipublixweeklyad.shop/
This week’s Save Mart Weekly Ad and next week’s Save Mart Ad! Go through every page of the Save Mart Weekly Circular. Prepare your coupons for the early Save Mart deals and schedule your shopping trip in advance!
This week’s Save Mart sales advertisement is really simple to peruse. To make it simple to determine if the item you are looking for at the Save Mart grocery shop is on sale or not, the sales are categorised.
https://savemart-weeklyad.shop/
This week’s Winn Dixie Sale Ad and the Winn Dixie Ad Preview for next week! To scroll through every page of the Winn Dixie Weekly Flyer, use the left and right arrows. I appreciate Sam alerting me to the early advertisement.
https://winndixieweeklyad.online/
This week’s Brookshires advertisement is quite simple to peruse. To make it simple to determine if the item you’re looking for at Brookshire’s grocery shop is on sale or not, the sales are categorized. To avoid overspending, make sure to check the weekly circular the next week to see if the item you wish to purchase is on sale.
https://brookshiresweeklyad.shop/
This week’s Super King advertisement poster is quite simple to peruse. To make it simple to determine if the item you are looking for at the Super King grocery shop is on sale or not, the sales are divided into different categories. Regardless of the item you like to purchase, be sure to check the Super King weekly advertisements for next week to see whether it is on sale so that
https://superking-weeklyad.shop/
The Schnucks weekly flier is quite simple to peruse. To make it simple to determine whether or not the product you are looking for is on sale during this week’s Schnucks sales, the sales are divided into categories.
Look through the Schnucks advertisement for next week and take advantage of the greatest offers to save a ton of money! Don’t pass up the fantastic discounts in the next Schnucks weekly advertisement!
https://schnucks-weeklyad.shop/
This week’s Stop and Shop Circular and next week’s Stop and Shop Flyer Go through every page of the weekly advertisement for Stop and Shop.
Prepare your coupons for the early Stop & Shop Weekly Circular and schedule your shopping excursion in advance!
https://stopendshopweeklyad.shop/
This week’s Food City Weekly Ad and next week’s Food City Ad! To view every page of the Food City weekly ad circular, use the left and right arrows. Prepare your coupons for the early Food City weekly ad preview and schedule your shopping trip in advance!
https://foodcityweeklyad.info/
This week’s Smart and Final Weekly Ad and next week’s Smart and Final Ad! To travel through every page of the Smart and Final weekly ad circular, use the left and right arrows. Prepare your coupons for the early Smart and Final weekly ad preview and plan your shopping trip in advance!
https://smartandfinalweeklyad.us/
This week’s Ingles Weekly Ad and next week’s Ingles Ad! To go through every page of the Ingles weekly ad circular, use the left and right arrows.
Prepare your coupons for the early Ingles weekly ad preview and schedule your shopping trip in advance. Make sure you are seeing all of the new Ingles weekly promotions by returning frequently.
https://inglesweeklyad.info/
Check out this week’s Publix Weekly commercial and the preview of next week’s Publix commercial! Examine every page of the Publix weekly ad preview. Depending on your store, the new advertisement begins on Wednesday or Thursday and continues for one week (check your business to see which schedule they use). Thank you for the photos, Savings 101!
https://ipublixweeklyad.shop/
This week’s Brookshire’s Ad, and next week’s Brookshire’s Weekly Ad! To travel through every page of the Brookshires Ad Flyer, use the left and right arrows.
Prepare your coupons for the early Brookshire’s weekly ad circular and schedule your shopping excursion in advance.
https://brookshiresweeklyad.shop/
This week’s Save Mart sales advertisement is really simple to peruse. To make it simple to determine if the item you are looking for at the Save Mart grocery shop is on sale or not, the sales are categorised.
https://savemart-weeklyad.shop/