GIFT-Eval-CTX Contextual Forecasting Leaderboard

We present GIFT-Eval-CTX, a benchmark for evaluating whether forecasting methods can convert verified future information into accurate numerical trajectories. GIFT-Eval-CTX contains 10,000 instances drawn from 20 GIFT-Eval datasets, covering 7 domains and 6 frequencies, with prediction horizons ranging from 30 to 900 steps. Each instance provides structured effects derived from the realized future: Macro effects describe horizon-wide behavior, while Micro effects specify localized changes. Links to 📄 Paper, 🖥️ Code, and 📊 Data.

Loading evaluation results…Ranked by Overall MASE, lower is better
Loading evaluation results…
—Click a score heading to sort
How are leaderboard scores calculated?

We use Mean Absolute Scaled Error (MASE) as our evaluation metric. MASE scales the mean absolute forecast error by the historical seasonal naive error. Lower values indicate better forecasting accuracy.

The leaderboard reports MASE normalized by Seasonal Naive and aggregated across groups, calculated as follows:

  1. Group instances by dataset, sampling frequency, and forecast term, then compute the arithmetic mean of each method's per-instance MASE within each group.
  2. Divide each method's group-average MASE by Seasonal Naive's group-average MASE for the same group.
  3. Take the geometric mean of these normalized group scores. Each group receives equal weight.

Overall includes all groups. Short, Medium, and Long include only groups for the corresponding forecast term.