Passing use_variate_attention=False through TimesFM3Forecaster.from_pretrained("google/timesfm-3.0-pytorch", device=..., use_variate_attention=False) produces a model whose outputs are bit-identical to the default (max |diff| exactly 0.0 over ~390k point and quantile values on multivariate inputs; timesfm 3.0.0, torch 2.13): the kwarg reaches the constructor path but the flag is consumed at module construction and the loaded architecture is unchanged. Setting the attribute post-load (fc.model.use_variate_attention = False) is also a no-op at forward time.
Suggestion: either honor the kwarg (rebuild/bypass the variate-attention blocks) or raise on unsupported architecture overrides in from_pretrained — a silent no-op is dangerous in ablation studies, where the two configurations are expected to differ and identical results can masquerade as a finding. Happy to provide a minimal repro script.
Two smaller notes from a ~30k-forecast walk-forward on Apple MPS, if useful for docs: per-shape kernel recompilation makes naively growing context lengths ~8x slower than bucketing lengths to a fixed grid; and TimesFM_2p5_200M_torch only knows cuda/cpu at init — MPS works well (matches CPU to 1e-7) but requires setting model.device and moving the module manually, plus float32 inputs.
(Found while running a pre-registered release-day evaluation on the SPY implied-volatility surface: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=7406218, repro at https://github.com/charlieyanhx/market-already-knew.)
Passing
use_variate_attention=FalsethroughTimesFM3Forecaster.from_pretrained("google/timesfm-3.0-pytorch", device=..., use_variate_attention=False)produces a model whose outputs are bit-identical to the default (max |diff| exactly 0.0 over ~390k point and quantile values on multivariate inputs; timesfm 3.0.0, torch 2.13): the kwarg reaches the constructor path but the flag is consumed at module construction and the loaded architecture is unchanged. Setting the attribute post-load (fc.model.use_variate_attention = False) is also a no-op at forward time.Suggestion: either honor the kwarg (rebuild/bypass the variate-attention blocks) or raise on unsupported architecture overrides in
from_pretrained— a silent no-op is dangerous in ablation studies, where the two configurations are expected to differ and identical results can masquerade as a finding. Happy to provide a minimal repro script.Two smaller notes from a ~30k-forecast walk-forward on Apple MPS, if useful for docs: per-shape kernel recompilation makes naively growing context lengths ~8x slower than bucketing lengths to a fixed grid; and
TimesFM_2p5_200M_torchonly knows cuda/cpu at init — MPS works well (matches CPU to 1e-7) but requires settingmodel.deviceand moving the module manually, plus float32 inputs.(Found while running a pre-registered release-day evaluation on the SPY implied-volatility surface: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=7406218, repro at https://github.com/charlieyanhx/market-already-knew.)