# Time Series Foundation Models (TSFM)

In this section, we highlight the papers, blogs, pre-trained models, and open-source codes from IBM Research's TSFM group.

## HuggingFace releases

### Pre-trained models
1. **TinyTimeMixer (TTM):** https://huggingface.co/ibm-granite/granite-timeseries-ttm-v1  
🚀 _**Downloads: 1.1 Million+, Likes: 148**_ (as of 26 Aug 2024) 🚀

### Architectures
1. **PatchTSMixer:** https://huggingface.co/docs/transformers/en/model_doc/patchtsmixer

1. **PatchTST:** https://huggingface.co/docs/transformers/en/model_doc/patchtst


## Publications

4 KDD, 1 ICLR, 2 AAAI, 1 ICML.  
🚀 _**Total citations (Google Scholar): 1700**_ (as of 26 Aug 2024). 🚀

1. **TST:** Zerveas, G., Jayaraman, S., Patel, D., Bhamidipaty, A., & Eickhoff, C. [_A transformer-based framework for multivariate time series representation learning._](https://arxiv.org/abs/2010.02803) In KDD 2021. **(citations: 840)**

1. **PatchTST:** Nie, Y., Nguyen, N. H., Sinthong, P., & Kalagnanam, J. (2022). [_A Time Series is Worth 64 Words: Long-term Forecasting with Transformers._](https://arxiv.org/abs/2211.14730) ICLR 2023. **(citations: 656)**

1. **PatchTSMixer:** Ekambaram, V., Jati, A., Nguyen, N., Sinthong, P., & Kalagnanam, J. [_TSMixer: Lightweight MLP-Mixer Model for Multivariate Time Series Forecasting._](https://arxiv.org/abs/2306.09364) In KDD 2023. **(citations: 59)**

1. **NPF:** Ekambaram, V., Manglik, K., Mukherjee, S., Sajja, S. S. K., Dwivedi, S., & Raykar, V. [_Attention based multi-modal new product sales time-series forecasting._](https://dl.acm.org/doi/10.1145/3394486.3403362) In KDD 2020. **(citations: 68)**

1. **TLAE:** Nguyen, N., & Quanz, B. [_Temporal latent auto-encoder: A method for probabilistic multivariate time series forecasting._](https://arxiv.org/abs/2101.10460) AAAI 2021. **(citations: 67)**

1. **HPRO:** Jati, A., Ekambaram, V., Pal, S., Quanz, B., Gifford, W.M., Harsha, P., Siegel, S., Mukherjee, S. and Narayanaswami, C. [_Hierarchical proxy modeling for improved HPO in time series forecasting._](https://dl.acm.org/doi/abs/10.1145/3580305.3599378) In KDD 2023. **(citations: 6)**

1. **AutoMixer:** Palaskar, S., Ekambaram, V., Jati, A., Gantayat, N., Saha, A., Nagar, S., Nguyen, N.H., Dayama, P., Sindhgatta, R., Mohapatra, P. and Kumar, H. [_Automixer for improved multivariate time-series forecasting on business and it observability data._](https://ojs.aaai.org/index.php/AAAI/article/view/30336) In AAAI 2024. **(citations: 3)**

1. **ConCerNet:** Zhang, W., Weng, T.W., Das, S., Megretski, A., Daniel, L. and Nguyen, L.M. [_ConCerNet: A Contrastive Learning Based Framework for Automated Conservation Law Discovery and Trustworthy Dynamical System Prediction._](https://proceedings.mlr.press/v202/zhang23ao/zhang23ao.pdf) In ICML 2023. **(citations: 1)**

## Preprints

1. **TTM:** Ekambaram, V., Jati, A., Nguyen, N.H., Dayama, P., Reddy, C., Gifford, W.M. and Kalagnanam, J., [_Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Series._](https://arxiv.org/abs/2401.03955), arXiv preprint 2024.

1. Trang H. Tran, Lam M. Nguyen, Kyongmin Yeo, Nam Nguyen, Dzung Phan, Roman Vaculin, Jayant Kalagnanam. [_An End-to-End Time Series Model for Simultaneous Imputation and Forecast._](https://arxiv.org/abs/2306.00778) arXiv preprint 2023.

1. Anh Duy Nguyen, Trang H. Tran, Hieu H. Pham, Phi Le Nguyen, Lam M. Nguyen. [_Learning Robust and Consistent Time Series Representations: A Dilated Inception-Based Approach._](https://arxiv.org/abs/2306.06579) arXiv preprint 2023.


## Workshops/Invited Talks/Tutorials

1. Arindam Jati, Vijay Ekambaram, Pankaj Dayama, Nam H. Nguyen,Jayant Kalagnanam. [_Light-Weight Pre-Trained Mixer Models For Effective Transfer Learning In Multivariate Time Series Forecasting_](https://whova.com/embedded/session/a1FSCBNUVSoDe3YGmt0K2B2OQAm5arkz%401l4TJaUYuc%3D/3894088/?widget=primary). Presented at the 44th International Symposium on Forecasting (ISF), 2024, held at Dijon, France.

1. Sumanta Mukherjee, Chandramouli Kamanchi, Pankaj Dayama, Vijay Ekambaram, Arindam Jati, Kameshwaran Sampath. [_Intervention Aware Forecasting For Process Control With Sparse Data_](https://whova.com/embedded/session/a1FSCBNUVSoDe3YGmt0K2B2OQAm5arkz%401l4TJaUYuc%3D/3894486/?widget=primary). Presented at the 44th International Symposium on Forecasting (ISF), 2024, held at Dijon, France.

1. Lam M. Nguyen, Trang H. Tran, Wang Zhang, Subhro Das, Tsui-Wei Weng. _When Machine Learning meets Dynamical Systems: Theory and Applications_. Workshop at The 37th Conference on Artificial Intelligence (AAAI 2023).

## Blogs and News Articles

We sincerely thank all the blog authors for dedicating their valuable time to analyzing and exploring our TSFM models. The analysis and conclusions presented are entirely the work of the respective authors.

### IBM Research blogs
- **June 4, 2024**: [At Think, IBM showed how generative AI is set to take automation to another level](https://research.ibm.com/blog/automation-think-generative-ai)

- **March 14, 2024**: [Generative AI could offer a faster way to test theories of how the universe works](https://research.ibm.com/blog/time-series-AI-transformers)

- **February 1, 2024**: [A crystal ball made of AI transformers](https://research.ibm.com/blog/AI-time-series-forecasting)

- **February 1, 2024**: [Patch Time Series Transformer in Hugging Face - Getting Started](https://huggingface.co/blog/patchtst) (joint blog post with Hugging Face)

- **January 19, 2024**: [PatchTSMixer in HuggingFace - Getting Started](https://huggingface.co/blog/patchtsmixer) (joint blog post with Hugging Face)

- **June 26, 2023**: [TS Foundation Models - The Battle of Time-series Transformers](https://www.linkedin.com/pulse/ts-foundation-models-battle-time-series-vijay-ekambaram/)


### From External Authors


- **August 23, 2024**: [IBM reveals why its "tiny" AI models punch well above their weight](https://www.thestack.technology/ibm-tiny-time-mixer-ai/)
  
- **August 19, 2024**: [‘Tiny’ AI, big world: New models show smaller can be smarter](https://www.fierce-network.com/cloud/tiny-ai-big-world-ibms-new-model-shows-smaller-can-be-smarter)

- **July 31, 2024**: [Tiny Time Mixers (TTMs) for Next-Level Time Series Forecasting](https://medium.com/@yi.angela/tiny-time-mixers-ttms-for-next-level-time-series-forecasting-5aa07365d963)

- **July 17, 2024**: [Exploring the Latest Advances in Foundation Time-Series Models](https://towardsdatascience.com/exploring-the-latest-advances-in-foundation-time-series-models-3fc8431ab7bd)

- **June 11, 2024**: [IBM showcased GenAI’s potential to reshape business automation at Think 2024](https://indiaai.gov.in/article/ibm-showcased-genai-s-potential-to-reshape-business-automation-at-think-2024)

- **June 4, 2024**: [Tiny Time Mixers(TTMs): Powerful Zero/Few-Shot Forecasting Models by IBM](https://aihorizonforecast.substack.com/p/tiny-time-mixersttms-powerful-zerofew)

- **May 14, 2024**: [Predicting Venetian Lagoon Tide Levels with Multivariate Time Series Modeling](https://medium.com/@david.proietti_17/predicting-venetian-lagoon-tide-levels-with-multivariate-time-series-modeling-8bafdf229588)




- **June 20, 2023**: [PatchTST: A Breakthrough in Time Series Forecasting](https://towardsdatascience.com/patchtst-a-breakthrough-in-time-series-forecasting-e02d48869ccc)

- **May 25, 2023**: [The Return of the Fallen: Transformers for Forecasting](https://towardsdatascience.com/the-return-of-the-fallen-transformers-for-forecasting-24f6fec5bc30)

- **May 17, 2023**: [PatchTST for Time Series Forecasting: Original Results and My Single-Channel Experiments](https://medium.com/@lalf_klein/patchtst-for-time-series-forecasting-original-results-and-new-single-channel-experiments-f375699f7b91)



## Tutorial videos

We sincerely thank all the tutors for dedicating their valuable time to analyzing and exploring our TSFM models. The analysis and conclusions presented are entirely the work of the respective authors and tutors.

1. [TinyTimeMixer TTM Model by IBM - Run in Google Colab for Forecasting (July 5, 2024)](https://www.youtube.com/watch?v=83j5FddZBNs)

1. [PALS Industry Assisted Lecture Series on AI for IOT: Introduction and Hands-On AI for IOT Data (Oct 12, 2023)](https://www.youtube.com/watch?v=B9IFffk1JMc&t=4906s)

<!-- 1. [PatchTST — A Step Forward in Time Series Forecasting (June 24, 2023)](https://pub.towardsai.net/patchtst-a-step-forward-in-time-series-forecasting-13a8e8f53feb) -->

## Model-wise List of Opensource Implementations

| Model   | Repository   | Stars | Forks | Comment |
|------------|------------|------------|------------|------------|
| TTM   | [granite-tsfm](https://github.com/ibm-granite/granite-tsfm)   | 319 | 160 | From the authors   | 
| TTM   | [sktime](https://www.sktime.net/en/latest/api_reference/auto_generated/sktime.forecasting.ttm.TinyTimeMixerForecaster.html) | -- | -- | -- |
| PatchTSMixer | [HuggingFace](https://huggingface.co/docs/transformers/en/model_doc/patchtsmixer) | -- | -- | From the authors |
| PatchTST | [HuggingFace](https://huggingface.co/docs/transformers/en/model_doc/patchtst) | -- | -- | From the authors |
| PatchTST | [GluonTS](https://ts.gluon.ai/stable/api/gluonts/gluonts.torch.model.patch_tst.html) | -- | -- | -- |
| PatchTST | [Nixtla](https://nixtla.github.io/neuralforecast/models.patchtst.html) | -- | -- | -- |
| PatchTST | [yuqinie98/PatchTST](https://github.com/yuqinie98/PatchTST) | 1.5k | 256 | From the authors |
| TST | [tsai](https://timeseriesai.github.io/tsai/models.tst.html) | -- | -- | -- |
| TST | [mvts_transformer](https://github.com/gzerveas/mvts_transformer) | 734 | 171 | From the authors |
