Cheatsheet — the whole project in one page

a short read for someone new to grid forecasting · what the problem is, what we found, and the vocabulary

The problem in one paragraph

The US grid is split into ~66 control areas called balancing authorities (CAISO, PJM, ERCOT, …). Each has to match electricity generation to electricity demand in real time. To schedule generators economically, they need to know tomorrow's hourly demand today — a day-ahead load forecast. Errors are expensive both ways: under-forecast and you scramble for peaking plants (or shed load); over-forecast and you hold costly spinning reserves that burn fuel for nothing. Even a fraction of a percent of error compounds into millions of dollars and megatons of CO₂ across a year.

The result, in three lines

What we actually built (not an off-the-shelf model)

The model architecture is in the iTransformer family, but the substantive contributions live around it. The result number depends on every one of these:

1. A permutation-invariant adaptation of the iTransformer. Each BA becomes a cross-attention token; instance-norm + quantile loss + static BA features form a recipe that generalizes to BAs never seen in training. That's what makes one-shot evaluation on 7 held-out BAs honest — and what foundation models structurally can't do.
2. A scaling sweep specific to this task. 4×4 size×data sweep (18 cells) fit with the Chinchilla-style additive form. Directional, not Kaplan-grade — the sweep spans ~2 orders of magnitude in N and ~1 in D, which isn't enough to calibrate α, β, or E∞ tightly. What's robust: at the sizes we tested, the parameter axis is dead and the data axis still has slope. The FERC-714 XBRL extension (+29% data → −0.014pp MAPE) hit that direction, validating "data still pays" as a finding even though the sweep can't validate β as a number.
3. FERC-714 XBRL data recovery. We wrote an XBRL parser to pull utility-level hourly demand from the messy FERC filings back to 2010 — recovering the only piece of in-distribution data that measurably moved the metric, exactly as the scaling law predicted.
4. Anti-pollution discipline enforced in code. Assert gates in every loader, pretest-only normalization, one-shot temporal split, multi-seed paired-bootstrap on every headline gain. This protocol — not the model — is what caught BA-mixup, the L80 over-claim, and the "MAPE is operationally useless" mirage.
5. An operational-value framework. A battery-dispatch simulator under two objectives (peak-shaving + price arbitrage on real ERCOT 2024 hub prices) that translates MAPE into the units operators actually care about. Refuted our own most exciting interim finding.
6. Controlled head-to-head re-running the competitor. We re-trained PatchTST and a plain iTransformer ourselves on the identical Hong & Lee 2026 split, with a faithfulness check (our PatchTST reproduces their published 3.66 to 0.02pp). The comparison stops depending on which published table their competitor number lived in. Against plain iTransformer (no perm-invariance, no instance-norm, no quantile loss) on this controlled ISO eval our recipe wins 3.52 vs 3.69 macro, CI [−0.196, −0.142] excludes zero — that's the closest direct ablation of "the recipe beyond an off-the-shelf iTransformer." For the headline held-out-BA task, plain iTransformer can't even run (positional embeddings tie each variate to a position; no slot for a never-seen BA at test time) — perm-invariance there is a prerequisite, not an ablation knob.
7. An EIA-930 telemetry-artifact audit. Caught that LDWP's "3% MAPE" was the meter, not the model — pretest-derived cleaning mask, documented as a dataset issue not a modeling one.
8. A documented graveyard. 13 negatives — BA-mixup, MoE, foundation-model stacks, GridLAB-D synth v1/v2, pre-2018 EIA, interchange + fuel-mix channels, weather covariates — each with a why. Negative results are the rare-to-publish artifact and the most useful single output of this project for anyone working the same problem.

The model alone is ~200 lines of PyTorch. The other 90% of the work is what makes the 1.02% defensible.

How it's measured

The headline metric is MAPE on observed-positive hours, macro-averaged across the 7 held-out BAs, on a one-shot forward window (Nov 16 2024 + 2025-Feb–Dec). 8 BAs are entirely absent from training, so the metric measures genuine out-of-distribution generalization — not memorization on data the model has seen.

How we estimated the floor — scaling sweep, not a Kaplan-grade law

We ran an 18-cell scaling sweep at 4 model sizes × 4 data scales (16 cells at 20 epochs) + 2 compute-corrected 40-epoch cells. To each set of 18 MAPE numbers we fit the Chinchilla-style additive form MAPE(N, D) = E∞ + A·N⁻ᵅ + B·D⁻ᵝ. The fit gives E∞ ≈ 0.85% (range 0.84–0.91 across slice fits) — a suggestive floor, not a calibrated one. The shape of the surface is what's load-bearing here, not the parameter values.

Each line is one model size across data scales · ★ = 40-epoch (compute-corrected) cell · dashed red = fitted floor E∞ ≈ 0.85% — directional, not a calibrated CI.

Two readings the picture supports (qualitatively, not quantitatively):

• Model-size axis is dead at the sizes we tested. L (2.2M) ties or beats XL (7M) at 100% data on 20 ep — once you train at matched compute, the bigger model buys nothing in our regime. (Honest caveat: this might just mean XL needs re-tuned LR + warmup. We did not re-tune per-N.)
• Data axis still has slope. Every model size's MAPE drops monotonically as data grows. β ≈ 0.5 falls out of the fit, broadly consistent with Chinchilla-flavored time-series literature; the FERC-714 XBRL extension (+29% data → −0.014pp MAPE) hits the projected direction, but the test is too small to validate β as a number.

Why we test on two dispatch modes

The "does accuracy matter" claim is much stronger if it holds under two different uses of the forecast — they stress it in different ways. We test both:

Different sensitivities: peak-shaving lives or dies on one hour's timing; arbitrage lives on the whole-day shape. If the forecast captured value under only one we'd have to caveat ("accuracy pays for this use case"). It captures 86–89% under both, so the claim "MAPE translates to operational value" is robust to which dispatch problem you actually care about.

Per-model dollar savings — three battery durations

We ran the dispatch sim across the foundation-model forecasts we'd already trained, at three battery durations, and converted to dollars per MW of installed battery per year. Tariff: $15/kW-month demand charge (midpoint of PG&E E-19 and ConEd SC9-II). Arbitrage: real ERCOT-2024 day-ahead-market hub prices. Why three durations: 2h dominates installed C&I storage today, 4h is the FERC Order 841 / CAISO RA reference, 8h is the long-duration / utility-scale regime.

Bench: 49 EIA-930 BAs ex-LDWP · 2025 Feb-Dec · 15,974 BA-days × 3 durations · 10% of mean BA demand · branch feature/dispatch-bench.

$/MW-year is the academic-standard unit but it's the wrong unit for "is this a lot." Anchored to actual fleet sizes at the 4-hour reference:

Operational takeaway in absolute dollars: GridFloor adds ~$25k (2h) / ~$47k (4h) / ~$49k (8h) per MW-year over persistence on the 2 revenue streams we modeled, holding roughly constant in absolute dollars as duration grows even as percent-of-perfect collapses. At US grid-scale fleet size that's ~$2.4B/year over persistence and ~$1.3B/year over the best foundation model — and that's before adding the 3 unmodeled revenue streams where forecast accuracy matters more.

Glossary

Why this matters in the real world

It's the metric utilities actually run against. Grid operators commit generation a day ahead against the load forecast. A 1.02 vs 9.04 gap on a 100 GW grid is ~1 GW less missed per hour — about one large power plant. Lower error means fewer reserve plants kept "just in case" → lower cost, lower emissions. The dispatch finding shows the gain isn't academic: 86–89% of the value perfect foresight would capture, above the 24h-persistence floor, under both peak-shaving and price arbitrage.

It bounds the field. The scaling-law floor at E∞ ≈ 0.85% says no amount of extra parameters or epochs gets lower. The remaining 0.1pp is a data-acquisition question, not a modeling one — and same-distribution data is nearly exhausted. So the honest next direction is structural (sub-BA zonal expansion, renewable-gen as a parallel target) rather than bigger models.

What's still open

• Close the last 0.1pp to the floor — would require more in-distribution data: sub-BA zonal series, careful handling of AEMO / ENTSO-E distribution shift.
• Per-region renewable forecasting — same cross-region structure should help; weather is the upstream driver, a perfect candidate for the decomposed approach.
• Net-load / duck-curve forecasting — the operationally-critical one for dispatch scheduling as solar penetration grows.
• Calibrated probabilistic intervals — point MAPE is solved; sharpness + coverage on the desert-Southwest BAs is genuinely hard (interval width is bounded by point error).