Understanding data with AI
All kinds of patterns can be found in data, some of which are obvious to humans while others can only be teased out by advanced machine learning algorithms. Some of the most interesting, and difficult to discern, patterns are those that span across massive datasets that, once identified show how seemingly disparate ideas can share a common structure. Similarities across data types is something that AI can be help identify and take advantage of, allowing for unconventional applications of AI across different systems that might otherwise seem unrelated.
In a recent podcast, Dr. James Loach, head of research at Senseye Predictive Maintenance, explained how, at a fundamental level, time series foundation models and text generation models can function similarly and how that was key for development of anomaly forecasting models within Senseye. James also explores some of the challenges of creating these models and why they were worth the effort.
Check out the full episode here or keep reading for a summary of that conversation.
Time vs. text as tokens
To a human, the idea comparing time series data to textual information seems absurd and in the context of the way the human brain processes information it doesn’t make much sense, but for an LLM, the story changes. Foundation models, and by extension, the LLMs that are built on them, work by predicting what the next word in a sentence should be in the form of tokens. Based on a given query, the model constructs a response one token at a time, using advanced algorithms to build up each subsequent word in a sentence. James goes on to explain how the same approach can be used with time series data by treating a time series as a language with an “infinite” vocabulary.
Using the previous time series data up to the current time as an input, a time series foundation model could forecast where the data is going extremely accurately using the exact same token generation method an LLM would use to answer a question, enabling fast and precise predictions. Compared to other methods of data fitting, the time series foundation model method has the advantage of intelligence as well, allowing it to follow trends in the data without triggering false alarms when they turn out to be nothing and learning ahead of time precursors to critical events that might go unnoticed by the human eye.
Building a time series foundation model
From a machine perspective, time series and language data might be seen interchangeably, actually building a model that can take advantage of that presents a unique set of challenges. The biggest challenge James highlights is one of vocabulary; language as a whole as a finite number of words in it while a time series can be divided into an potentially infinite number of points, effectively creating an infinite vocabulary. A key challenge they had to overcome was finding the best way to discretize time series data to train the model with since many different approaches exist.
Picking the right approach was key for Senseye to develop a useful model, allowing the model to learn to follow trends in the data when needed without overreacting or under reporting, depending on what kinds of trends were being presented. In the end, Senseye chose to build a model that focused on following trends and looking at time series predictions as a continuous function, rather than simply trying to predict the next point of data. James says this approach has worked well enough for both them and their customers that they consider it to be a solved problem.
Creating a model that could handle was both the challenge and the key to offering a seamless experience to end users, a tool that works exactly as expected with a far greater degree of intelligence than older methods offered to the point where the AI model can simply be trusted to function. Senseye’s success with the time series foundation model shows a glimpse at what the future of AI has to offer, a seamless intelligence layer that helps users make the most of their data and accurately inform their most important decisions.
Siemens Digital Industries Software helps organizations of all sizes digitally transform using software, hardware and services from the Siemens Xcelerator business platform. Siemens’ software and the comprehensive digital twin enable companies to optimize their design, engineering and manufacturing processes to turn today’s ideas into the sustainable products of the future. From chips to entire systems, from product to process, across all industries. Siemens Digital Industries Software – Accelerating transformation.
