After the fierce competition over large models since 2023, this trend hasextended its reach to the field of time series.

This article aims to answer the most pressing questions about time series large models: the evolutionary path of time series data analysis, the key players in the time series large model arena, and the effectiveness of these models in practical applications.

Evolutionary Path

Time series large models are undeniably an emerging field.

Before 2018, the industry primarily relied on statistical models for time series data analysis. These approaches included classical techniques such as autoregressive moving average models (ARMA), trend analysis, state-space models (SSM), and spectrum analysis. These methods used mathematical formulas to describe relationships between time points, uncover underlying patterns, or model time series generation processes.

Since then, research on deep learning models for time series has flourished, leading to two major categories of advancements:

• General Time Series Analysis: Rooted in classical statistical methods, these approaches remained foundational but began to integrate computational advances for greater precision.

• Deep Forecasting Models: Leveraging neural networks, these models addressed challenges like modeling large-scale data, managing complex temporal variations, and handling multivariate relationships. Despite their advancements, they struggled with the diversity and non-stationarity of time series data, as well as limitations in generalization and adaptability across tasks.

In 2024, a new branch of models emerged—time series large models. These advanced models represented breakthroughs in versatility, adaptability to downstream tasks, scalability, and predictive accuracy. Current research focuses on two main directions:

• Applying Large Language Models (LLMs): Techniques from natural language processing (NLP) are adapted to time series, leveraging exceptional pattern recognition and prediction capabilities.

• Developing Native Time Series Large Models: Tailored specifically for time series data, these models incorporate attributes like continuity, periodicity, and time dependency for more accurate and efficient analysis.

Typical Achievements

Numerous globally influential companies have played indispensable roles in the development of time series large models. Here are some prominent examples:

• TimesFM: Open-sourced by Google, this pre-trained decoder-only foundation model is optimized for predicting univariate time series with a context length of up to 512 time points across any time span.

• MOIRAI: Open-sourced by Salesforce, this cutting-edge model offers universal forecasting capabilities in a zero-shot manner.

• Chronos: Open-sourced by Amazon, it tokenizes time series data and trains language models using tokenized sequences and cross-entropy loss functions.

• TimeGPT-1: Developed by Nixtla, it incorporates local positional encoding to enrich input and supports fine-tuning with user-specific data through a REST API endpoint.

Universities and individual developers have also contributed significantly:

• MOMENT: A pre-trained model designed for masked sequence prediction tasks using large-scale public time series datasets.

• Lag-Llama: Built for univariate probabilistic forecasting, leveraging prior knowledge like data lags to improve generalization.

• ForecastPFN: Focused on zero-shot time series forecasting, it uses synthetic data with predefined trends and seasonality to analyze future trends.

Researchers at universities have made notable advancements as well, for instance, the Timer model. Originating from the School of Software at Tsinghua University, where the time series database IoTDB was initiated, the Timer model is specifically designed for the field of time series. Designed for time series applications, Timer leverages large-scale pre-training to offer remarkable generalization, scalability, and prediction capabilities.

Currently, the Timer model is integrated into the AINode of IoTDB, allowing users to conveniently access it within IoTDB. Thanks to the Timer model's outstanding performance, IoTDB can effectively provide solutions for industrial scenarios such as time series forecasting, data imputation, and anomaly detection.

The key features of the aforementioned time series large models, including their structure, model characteristics, and supported application scenarios, are summarized in the following table:

Performance Comparison

A typical scenario for evaluating time series large models is zero-shot prediction tasks, which test a model’s ability to make predictions on previously unseen datasets. It is challenging to train models solely based on historical time series data because of its massive volumes and high write concurrency. As a result, handling zero-shot prediction tasks requires the model to have strong abilities of feature extraction, and the performance of these tasks could directly reflect a model's generalization capabilities.

The figure below presents a comparison of zero-shot prediction performance among various large models across 7 real-world datasets. The metric used is Mean Squared Error (MSE), where a lower value indicates smaller differences between predicted and actual values, signifying higher prediction accuracy.

The results highlight Timer’s consistent ranking among the top three models across multiple datasets. In terms of average ranking—calculated as the proportion of datasets where a model achieves the top position—Timer demonstrates superior overall performance.

Beyond zero-shot prediction, Timer excels in few-shot fine-tuning, task versatility, scalability, and adaptability to variable sequence lengths.

Conclusion

With the concentrated release of research breakthroughs, time series large models are developing rapidly, demonstrating immense potential across various domains and applications.

As technology continues to advance, we anticipate the emergence of innovative methods and practices, driving significant performance improvements. The Timer model, for example, exemplifies these advancements, setting benchmarks for time series analysis. The future holds exciting possibilities for furthering the capabilities of time series large models, empowering them to solve increasingly complex and diverse real-world challenges.