Advanced Time Series Analysis: From probabilistic to foundational models

Advanced Time Series Analysis: From Probabilistic to Foundational Models

Time series analysis studies data that change as a function of time, such as stock market prices, weather patterns, or household electricity consumption.  This seminar covers advanced techniques for analyzing time series, starting with probabilistic methods and progressing to state-of-the-art deep learning approaches, including neural architectures and foundation models. We will also explore connections between time series and other modalities, such as text and images/videos, to offer a comprehensive view of the field. The aim is for students to critically assess existing methods, understand their strengths and limitations, and identify potential directions for future research.

The course begins with three introductory lectures to establish foundational concepts, followed by three main blocks on time series analysis, covering

• Block 1: Explicit Modeling of Time (Traditional Approaches)

• Block 2: Implicit Modeling of Time (Data-Driven Approaches)

• Block 3: Foundation Models and Task-Agnostic Approaches

Each block includes two paper sessions and a panel discussion, as detailed below. Student evaluation is based on their presentations, active participation in discussions, and a final report summarizing the seminar topics, offering critical analysis, identifying limitations, and suggesting potential research directions.

 

Day	Block	Content
16/04/2025	Presentation	Short session on seminar organization
23/04/2025	Lecture 1	Introduction to Time Series Analysis
30/04/2025	Lecture 2	Probabilistic Foundations for Time Series I
07/05/2025	Lecture 3	Probabilistic Foundations for Time Series II
14/05/2025	Block 1 - Explicit Modeling of Time	2 Student Presentations + Q&A
21/05/2025	Block 1 - Explicit Modeling of Time	2 Student Presentations + Q&A
28/05/2025	Block 1 - Explicit Modeling of Time	Round Table
04/06/2025	Block 2 - Implicit Modeling of Time	2 Student Presentations + Q&A
11/06/2025	Block 2 - Implicit Modeling of Time	2 Student Presentations + Q&A
18/06/2025	Block 2 - Implicit Modeling of Time	Round Table
25/06/2025	Block 3 - Foundation Models and Task-Agnostic Approaches	2 Student Presentations + Q&A
02/07/2025	Block 3 - Foundation Models and Task-Agnostic Approaches	2 Student Presentations + Q&A
09/07/2025	Block 3 - Foundation Models and Task-Agnostic Approaches	Round Table

 

Date and time:  Weekly.  Wednesday 12:30-14:00

Location:  SR 4 (U16) in E 2.5

 

Pool of Papers

This selection of state-of-the-art papers represents what we believe are valuable resources for discussing the topics covered in this seminar. Students are welcome to suggest alternative papers they find relevant. Please submit any proposals to the seminar organizers for approval. Papers can be proposed until April 21st, and the final selection will be announced by April 23rd. Papers listed in bold are essential and must be presented in the seminar. 

1. Block 1: Explicit Modeling of Time (Traditional Approaches)

   • Modeling Interleaved Hidden Processes
   • Scaling Factorial Hidden Markov Models: Stochastic Variational Inference Without Messages 
   • FlowHMM: Flow-based continuous hidden Markov Models
   • WaveNet: A Generative Model for Raw Audio
   • DeepAR: Probabilistic forecasting with autoregressive recurrent networks

2. Block 2: Implicit Modeling of Time (Data-Driven Approaches)

   • Attention Is All You Need
   • Set functions for time-series
   • Latent ODEs for Irregularly-Sampled Time-Series
   • BRITS: Bidirectional Recurrent Imputation for Time Series
   • A Time Series is Worth 64 Words: Long-term Forecasting with Transformers
   • iTransformer: Inverted Transformers Are Effective for Time Series Forecasting 
   • Non-autoregressive Conditional Diffusion Models for Time Series Prediction 

3. Block 3: Foundation Models and Task-Agnostic Approaches

   • Unified Training of Universal Time Series Forecasting Transformers (MOIRAI)
   • BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
   • Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction
   • Align Your Latents: High-Resolution Video Synthesis with Latent Diffusion Models 

 

Paper Assignment

Paper assignments will be managed through this form, which all students are required to fill out. Each student must rate every paper using the following options: "I prefer not to present," "I would present," or "This paper is for me." At least five papers must be marked as "I would present."

Students are also welcome to suggest additional papers by including the title, a link, and a brief justification for their selection. If the TAs think the proposed papers are appropriate, students will get to present the paper they have chosen (as long as each block still includes four papers and all mandatory papers are covered).

The deadline to complete the form is April 21st. Students who do not submit their responses by then will be assigned a paper at random. Final assignments will be made by the TAs and communicated by April 23rd.

 

Deliverables and Grading Scheme 

• Paper Presentation (15-20 minutes) (40%)

  • Submission (requirement): Students should submit a pdf file with the slides the day they are presenting.
  • Context 
    • Positioning of the paper within the state of the art and identification of gaps the paper addresses. 
    • Clear articulation of What/Why/How. 
    • Connection to the seminar blocks, e.g. Explicit, Implicit, Task Agnostic modeling of time. 
  • Content 
    • Clear explanation of the paper's core intuition and methodology. 
    • Rationale behind experiments and significance of results. 
    • Advantages and disadvantages of the approach. 
    • Final slide/section presenting the take-home messages. 
  • Q&A 
    • Responding questions from TAs and audience.

• Discussion Session (20%)

  • Pre-Submission Requirements
    • Each participant must submit one discussion question per block.
    • Submission deadline: 2 days before the session (Monday) via CMS.
  • Participation Expectations
    • For Presenters:
      • Quality of pre-submitted questions.
      • Active engagement with questions. 
      • Facilitating broader discussion. 
    • For Listeners:
      • Quality of pre-submitted questions. 
      • Active participation in discussions. 

• Final Report (6-8 pages, excluding references) (40%)

  • Template: https://www.overleaf.com/read/jmrjkxzrxpjn#2eff89
  • Critical Analysis
    • Comprehensive summary of the seminar from the perspective of time series modeling approaches (implicit, explicit, and foundation models), including the key points from the roundtable sessions. 
    • Comprehensive analysis regarding limitations, advantages, and open challenges in the field. 
  • Evaluation Criteria
    • Content
      • Depth of the analysis.
      • Demonstration of understanding across all three seminar blocks.
      • Synthesis of seminar content with broader research context.
    • Delivery 
      • Quality of writing and argumentation.