Skip to main content
W&B Fully Connected
>
Articles
>
Beginner

TimesNet: Revolutionizing Time Series Analysis with 2D Tensor Transformation

This article delves into the TimesNet architecture, which transforms 1D time series data into 2D tensors, capturing complex temporal variations for tasks like forecasting and anomaly detection.
Tensor Girl
Created on December 12|Last edited on December 27
This paper focuses on modeling temporal variations in time series data, which is key for many time series analysis tasks like forecasting, anomaly detection etc. Previous methods that operate directly on the 1D time series struggle to capture complex temporal patterns. The paper proposes a new method called TimesNet to better model temporal variations.
Here's what we'll be covering ...
Central Ideas Introduced With TimesNetNet ArchitectureExperimentsConclusionWhat is the TimesNet Architecture?Transform 1D-Variations into 2D-VariationsWhat Is TimesBlock?Adaptive AggregationGenerality in 2D vision backboneCode ExampleTrain loss curve for runs


Let's dive in.

Central Ideas Introduced With TimesNet

  • Time series often exhibit multiple periodicities (daily, weekly etc). These interact in complex ways.
  • The variations can exist within a period (intraperiod) or between periods (interperiod).
  • 1D time series struggles to represent these two kinds of variations simultaneously.
  • The paper transforms the 1D time series into 2D tensors by having each column as a period and each row as time points across periods.
  • This embeds intraperiod and interperiod variations in the columns and rows, respectively.
  • The 2D representation allows better modeling of variations using 2D kernels.

Net Architecture

  • TimesNet uses a TimesBlock module to:
    • Discover multiple periods adaptively
    • Transform into 2D tensors based on periods
    • Model 2D variations using an inception-style block
  • Overall modular architecture: Each TimesBlock handles variations for one period.
  • Captures intricate patterns by disentangling multi-periodicity.

Experiments

  • Evaluated on five main time series analysis tasks:
    • Short and long-term forecasting
    • Imputation
    • Classification
    • Anomaly detection
  • Achieves new state-of-the-art results across all five tasks
  • Showcases TimesNet's effectiveness as a general-purpose time series modeling technique

Conclusion

  • Proposes modeling time series variations in 2D by multi-periodic transformation.
  • TimesNet leverages this using TimesBlocks to capture 2D variations.
  • Significantly advances state-of-the-art across diverse time series analysis tasks.
In totality, the paper introduces a novel technique to model complex time series variations by transforming them into 2D tensors based on multi-periodicity and using TimesNet to capture 2D variations modularly. Extensive experiments demonstrate its effectiveness.

What is the TimesNet Architecture?

The TimesNet architecture introduces a novel approach to time series analysis by transforming one-dimensional (1D) time series data into a two-dimensional (2D) tensor format. This transformation is designed to capture complex temporal variations more effectively.
For each distinct period present, TimesNet uses a custom module called TimesBlock to transform the one-dimensional time series data into a two-dimensional representation. This 2D transformation allows TimesBlock to jointly capture the variations within a period and variations between periods efficiently using an inception-style block with shared parameters. TimesNet leverages its modular architecture with periodicity-specific TimesBlocks to model complex temporal patterns effectively by converting 1D time series to 2D and learning intraperiod and interperiod variations simultaneously.

Transform 1D-Variations into 2D-Variations

Most prior time series analysis methods operate directly on the raw one-dimensional time series data. However, modeling the intricate temporal variations within this 1D series is inherently challenging. The variations present can be categorized into two main types - intraperiod variations and interperiod variations. Intraperiod refers to the patterns and changes within a single period cycle (e.g. day, week, year). Interperiod refers to the variations across different period cycles.
Capturing these two kinds of temporal variations simultaneously in the 1D series is problematic. To address this, the authors propose transforming the 1D time series into a 2D representation. The key idea is to treat each period as a column and each time step across periods as a row.
This 2D transformation embeds the intraperiod variations within each column and interperiod variations across columns into the two dimensions. The 2D view makes capturing complex interactions between multi-scale periodic patterns more feasible. Standard 2D convolutions can now model the variations jointly in both dimensions.
In summary, converting from 1D to 2D time series based on periodicity disentangles the intraperiod and interperiod variations into the two dimensions. This enables more effective modeling of intricate temporal patterns using 2D operations. The proposed technique is a core enabler for the TimesNet architecture.


What Is TimesBlock?

TimesBlock is the core module in the TimesNet architecture designed to model temporal variations corresponding to a specific periodicity in the time series data.
The first component of TimesBlock is periodicity discovery, which analyzes the input 1D time series to detect significant periodic patterns adaptively using autocorrelation.
The next component transforms the 1D series into a 2D representation based on the identified period, with the columns representing each period cycle and rows representing time steps across periods.
This 2D transformation embeds intraperiod variations within columns and interperiod variations across columns into the two dimensions separately.
After converting to 2D, an Inception-style block is applied to model the intricate variations in parallel at multiple time scales. It uses convolutions with different kernel sizes to capture short-term and long-term patterns simultaneously.
The output is a 2D feature map summarizing the sophisticated periodic patterns discovered. By stacking multiple TimesBlocks, each specializing in one periodicity, TimesNet can jointly model the complex multi-scale variations.
In summary, TimesBlock handles periodicity-specific modeling via 1D-to-2D transform and multi-scale feature extraction in a modular fashion, forming the core of TimesNet's capabilities.


Adaptive Aggregation

After applying multiple TimesBlocks to model the temporal variations corresponding to different periodicities, the next step is aggregating their outputs. A simple approach would be to concatenate the outputs along the channel dimension. However, different periodic patterns may not contribute equally across time series.
To account for this, an adaptive aggregation mechanism is proposed. Specifically, the output of each TimesBlock is split into segments. The importance of each TimesBlock's output for a given segment is dynamically calibrated through learned attention weights. This allows for selectively emphasizing the most relevant periodic representations for each region of the input time series.
The attention weights are also determined based on temporally local context instead of the entire series. This makes aggregation computationally efficient while providing flexibility to focus on different periods across segments.
In summary, adaptive aggregation uses segment-wise attention to combine the outputs of different TimesBlocks in a context-aware dynamic manner. This further enhances TimesNet's capabilities in discovering and leveraging the key periodicities.


Generality in 2D vision backbone

The core component within each TimesBlock module is responsible for modeling the 2D representations transformed from the 1D time series. For this 2D feature learning, a general-purpose architecture is designed without restrictions to any specific time series domain.
Our approach utilizes proven concepts from convolutional neural networks for 2D vision tasks. Specifically, we adopt the Inception-style block structure with parallel convolutional layers of varying kernel sizes. This captures multi-scale patterns efficiently.
The 2D modeling backbone is kept generic without customizations for time series data. This promotes wider applicability across diverse forecasting, classification, anomaly detection, and other tasks.
The feature learning component generalizes well across datasets by leveraging flexible 2D CNN architectures from computer vision research. The domain-specific knowledge is encoded separately in the periodicity-based transformations. This separation of concerns results in an adaptable and performant architecture.
In summary, TimesNet employs a general 2D vision backbone to learn features from the transformed representations without overspecialization. This enhances its versatility across different time series modeling applications.

Code Example

!pip install neuralforecast wandb -q

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

from neuralforecast.core import NeuralForecast
from neuralforecast.models import NHITS, NBEATS, TimesNet

from neuralforecast.losses.numpy import mae, mse

import wandb
wandb.login(API_KEY)

df = pd.read_csv('/content/drive/MyDrive/train.csv')
df = df.rename(columns={'sales':'y'})
df.head()

run = wandb.init(project="timesnet")
# create a wandb Artifact for each meaningful step
data = wandb.Artifact("data", type="train")

table = wandb.Table(dataframe=df)
data.add(table, "first")
run.log_artifact(data)
run.finish()

run = wandb.init(project="timesnet")
data_ = run.use_artifact("data:latest")
dir = data_.download()
dir

table = data_.get('first')
df = pd.DataFrame(data=table.data, columns=table.columns)
df = df.rename(columns={'date':'ds','family':'unique_id'})
df = df.drop(['store_nbr', 'onpromotion','id'],axis=1)
df.head()

df['ds'] = pd.to_datetime(df['ds'],utc= False)
df.head()

df.dtypes

run.finish()

from pytorch_lightning.loggers import WandbLogger
horizon = 96
wandb_logger = WandbLogger(project='timesnet')
models = [TimesNet(h=horizon,
                 input_size=2*horizon,
                 max_steps=50,logger=wandb_logger)]

nf = NeuralForecast(models=models, freq='H')

preds_df = nf.cross_validation(df=df, step_size=horizon, n_windows=2)

run.finish()

from pytorch_lightning.loggers import WandbLogger

def main():
    run = wandb.init(project = 'timesnet',name='sweep')
    wandb_logger = WandbLogger(project='timesnet')
    horizon = wandb.config.horizon
    n_windows = wandb.config.n_windows
    models = [TimesNet(h=horizon,
                 input_size=2*horizon,
                 max_steps=50,logger=wandb_logger)]
    nf = NeuralForecast(models=models, freq='H')
    preds_df = nf.cross_validation(df=df, step_size=horizon, n_windows=n_windows)
    run.finish()

sweep_configuration = {
    'method': 'bayes',  # random, grid or bayes
    'name': 'sweep-bayes',
    'metric': {'goal': 'minimize', 'name': 'loss'},
    'parameters':
    {
        'horizon': {'values': [24, 48,96]},
        'n_windows': {'values': [2, 3,4]},
     }
}
sweep_id = wandb.sweep(sweep=sweep_configuration,project='timesnet')
wandb.agent(sweep_id, function=main, count=10)

Train loss curve for runs


Run set
16


System usage curves for runs


Run set
16



References

Tags: Articles, Beginner

Created with ❤️ on Weights & Biases.

https://wandb.ai/tensorgirl/timesnet/reports/TimesNet-Revolutionizing-Time-Series-Analysis-with-2D-Tensor-Transformation--Vmlldzo2MjQyMTMw
Made with Weights & Biases. Sign up or log in to create reports like this one.
Iterate on AI agents and models faster. Try Weights & Biases today.
Sign upTry W&B now