Five short labs for the Kalman filter and real-time nowcasting
Work through the labs in order the first time. In each one, stop at the prediction before moving a control. Use the display to test that prediction, then explain what changed and what stayed fixed. Open the explanation only after you have an answer. The point is not to obtain a particular number; it is to understand why the model produces it.
A Single Release: Innovation, Gain, and Updated State
Suppose the model’s estimate of current activity has mean a and variance P. A new indicator reports y and has measurement-error variance H. Change the four inputs and follow the calculation from the surprise to the updated estimate.
Predict before using the controls. If the release equals the prior mean, y=a, should the updated mean move? Should uncertainty remain unchanged, increase, or fall?
viewof releasePriorMean = Inputs.range([-2,6], {value:2,step:0.1,label:"Prior mean a (percentage points)"})viewof releasePriorVariance = Inputs.range([0.05,2], {value:0.25,step:0.05,label:"Prior variance P"})viewof releaseValue = Inputs.range([-2,6], {value:1.2,step:0.1,label:"Released value y (percentage points)"})viewof releaseMeasurementVariance = Inputs.range([0.05,2], {value:0.75,step:0.05,label:"Measurement-error variance H"})
releaseUpdate = {const a = releasePriorMean;const P = releasePriorVariance;const y = releaseValue;const H = releaseMeasurementVariance;const v = y - a;const F = P + H;const K = P / F;const aPost = a + K * v;const PPost = (1- K) * P;return { a, P, y, H, v, F, K, aPost, PPost };}releaseTable = [ {Quantity:"Innovation v",Definition:"released value − prior mean",Value:formatSigned(releaseUpdate.v,2) }, {Quantity:"Innovation variance F",Definition:"prior variance + measurement variance",Value: releaseUpdate.F.toFixed(2) }, {Quantity:"Kalman gain K",Definition:"prior variance ÷ innovation variance",Value: releaseUpdate.K.toFixed(3) }, {Quantity:"Updated mean a⁺",Definition:"prior mean + gain × innovation",Value: releaseUpdate.aPost.toFixed(2) }, {Quantity:"Updated variance P⁺",Definition:"(1 − gain) × prior variance",Value: releaseUpdate.PPost.toFixed(3) }]Inputs.table(releaseTable)
md`The release is **${Math.abs(releaseUpdate.v) <1e-9?"equal to":Math.abs(releaseUpdate.v).toFixed(2) +" percentage points "+ (releaseUpdate.v<0?"below":"above")}** the model's expectation.The gain passes**${(100* releaseUpdate.K).toFixed(0)}% of that surprise** into the stateestimate. The posterior standard deviation is**${Math.sqrt(releaseUpdate.PPost).toFixed(2)}**, down from**${Math.sqrt(releaseUpdate.P).toFixed(2)}**.`
Try these comparisons.
Set y=a. Compare the updated mean with the updated variance.
Hold a, P, and H fixed while moving y. Watch which of v,F,K,a^+, and P^+ change.
Return y to 1.2 and increase H. Explain why the release moves the estimate by less even though the surprise has not changed.
NoteWhy an Expected Release Still Matters
The gain multiplies the innovation v=y-a; it is not a weight placed directly on y. When y=a, the innovation is zero and the mean does not move. The variance still falls because observing the expected value rules out other values that could have occurred.
Holding P and H fixed while moving y changes v and a^+, but it does not change F, K, or P^+. Those three quantities describe expected precision before the release’s realized value is known. Increasing H makes the measurement less precise, lowers K, and leaves more uncertainty after the update.
Missing Observations: Filtering Through a Data Gap
The local level model separates changes in the state, measured by Q, from measurement error, measured by H. The filter observes the gray points; the orange state is shown only so that its estimates can be evaluated. The filter starts from a=0 and P=4, so its first few gains are deliberately large.
Predict before using the controls. Holding H fixed, should a larger Q raise or lower the gain? Holding Q fixed, should a larger H raise or lower it? Then predict what happens to the estimate, its uncertainty band, and the first gain after fifteen missing observations.
The blue band is a_{\tau\mid\tau}\pm1.96\sqrt{P_{\tau\mid\tau}}. It widens when prediction adds uncertainty and narrows when an observation provides information.
gapSummary = {const firstBack = filterSimulation.find(d => d.tau===70);const beforeGap = filterSimulation.find(d => d.tau===54);const endGap = filterSimulation.find(d => d.tau===69);return { firstBack, beforeGap, endGap };}withGap?md`No gain is computed during periods 55–69 because there is no measurement update. The filtered variance rises from **${gapSummary.beforeGap.variance.toFixed(3)}** after period 54 to **${gapSummary.endGap.variance.toFixed(3)}** after period 69. The first observation back receives gain **${gapSummary.firstBack.K.toFixed(3)}**, compared with the steady-state value **${steadyState.K.toFixed(3)}**.`:md`The signal-to-noise ratio is **q = Q/H =${steadyState.q.toFixed(3)}**, and the steady-state gain is **K̄ = ${steadyState.K.toFixed(3)}**. Turn on the data gap to see the prediction steps that occur when no measurement is available.`
Try these comparisons.
Increase Q while holding H fixed. Compare the gain and the amount of short-run movement in the filtered estimate.
Restore Q, then increase H. Compare the gain and the smoothness of the estimate.
Keep the seed fixed and turn on the data gap. Check that the orange state and all observations outside the gap remain the same.
NoteWhy Q and H Pull the Gain in Opposite Directions
A larger Q says the state can move substantially between periods. Older estimates therefore become obsolete more quickly, so new observations receive more weight. A larger H says each observation is a noisier reading of the state, so new observations receive less weight.
During the gap, the local level model has no measurement update. Its predicted mean remains at the last filtered value, while each prediction adds Q to the variance. The first observation after the gap receives a larger gain because its prior is less precise. The random draws are identical with and without the gap; only data availability changes.
This is a block of missing observations in one series. It is not yet a full ragged edge, which contains several series with different reference periods and release schedules.
Reference Periods and Release Dates: Building a Data Vintage
Every cell has two dates. The reference period \tau says what the value describes; the release date r says when it became available. Move through the three release dates. The rows remain attached to the same economic periods while values appear or are revised.
Predict before changing the release date. When r advances, which parts of the table can change? Which date must remain attached to each row? Decide whether a revised value belongs to its revision date or to the period it describes.
selectedVintageIndex = vintageDates.indexOf(selectedVintage)vintageView = vintageLedger.map(row => {const current = row.values[selectedVintageIndex];const previous = selectedVintageIndex >0? row.values[selectedVintageIndex -1]:null;let status;if (current ===null) { status ="Not yet released"; } elseif (selectedVintageIndex ===0) { status ="Available at the first displayed vintage"; } elseif (previous ===null) { status ="First release"; } elseif (current !== previous) { status ="Revised"; } else { status ="Unchanged"; }return {Series: row.series,"Reference period τ": row.referencePeriod, ["Value available at r = "+ selectedVintage]: current ===null?"Missing":formatSigned(current,1) + row.unit,"What changed at this release date": status };})Inputs.table(vintageView)
vintageCounts = {const available = vintageLedger.filter( row => row.values[selectedVintageIndex] !==null ).length;const firstReleases = vintageView.filter( row => row["What changed at this release date"] ==="First release" ).length;const revisions = vintageView.filter( row => row["What changed at this release date"] ==="Revised" ).length;return { available, firstReleases, revisions };}selectedVintageIndex ===0?md`At **r = ${selectedVintage}**, ${vintageCounts.available} of the 3 cells are available. This is the baseline vintage for the comparison. Changing release dates changes the information set; it does not change the reference period written on a row.`:md`At **r = ${selectedVintage}**, ${vintageCounts.available} of the 3 cells are available. This release date adds **${vintageCounts.firstReleases} first release${vintageCounts.firstReleases===1?"":"s"}** and **${vintageCounts.revisions} revision${vintageCounts.revisions===1?"":"s"}**. Changing release dates changes the information set; it does not change the reference period written on a row.`
Trace the information flow.
Move from 31 July to 15 August. Identify the first release and the observation that remains unavailable.
Move to 18 September. Separate new observations from revisions to older observations.
Ask what would go wrong if the September values were inserted into the July vintage.
NoteWhat Changes When the Vintage Date Changes
The reference period \tau determines where an observation belongs in the economic model. The release date r determines whether that observation is available in a particular information set. A September revision to July retail sales still describes July; it must simply be absent from vintages dated before September.
Advancing r can fill a missing cell or revise a previously observed cell. Putting a later value into an earlier vintage creates look-ahead bias. A filter can skip rows marked missing, but it cannot reconstruct a release calendar that was never stored.
Monthly Data and Quarterly GDP: The Five Aggregation Weights
Quarterly GDP is a flow. In the monthly approximation used in the notes, five monthly growth rates enter with weights (1/3,\,2/3,\,1,\,2/3,\,1/3). Change the monthly values to see which months contribute most to the quarterly rate.
Predict before using the controls. If each monthly growth rate were raised by 0.1 one at a time, which change would move quarterly growth the most? The five weights sum to 3 rather than 1. Decide whether they should be interpreted as probabilities.
md`The five weighted contributions sum to **quarterly growth =${formatSigned(quarterlyGrowth,2)}**. The middle month, τ−2, hasweight 1 because it enters every monthly-level comparison used to form thechange between adjacent quarterly averages.`
Try these comparisons.
Raise g_{\tau-2} by 0.1 and record the change in quarterly growth.
Return it to its original value, then raise g_\tau by 0.1. Explain why the effect is only one third as large.
Set all five monthly growth rates equal. Compare the monthly rate with the resulting quarterly rate.
NoteWhy the Aggregation Weights Form a Tent
Quarterly GDP compares averages of monthly levels in adjacent quarters. The middle monthly growth rate affects all three overlapping level comparisons, so it receives weight 1. The neighboring growth rates enter fewer comparisons and receive weights 2/3 and 1/3.
The weights are accounting weights, not probabilities; they need not sum to one. If monthly growth is constant at c, the approximation produces quarterly growth of 3c. This construction is appropriate for a flow. A stock measured at the end of the quarter requires a different rule.
News Decomposition: Which Release Moved the Nowcast?
A release changes a target through two numbers: its innovation v_i and the weight w_i mapping that surprise into the target. Their product is the release’s contribution. A separate control records a change caused by re-estimating the model rather than by data news.
Predict before using the controls. Can a positive innovation lower the GDP nowcast? What must be true of its target weight? Then decide whether a nowcast change caused by parameter re-estimation should be attributed to the latest data release.
md`Data news changes the nowcast by**${formatSigned(dataNewsRevision,2)} percentage points**. Parameterre-estimation contributes **${formatSigned(parameterRevision,2)}**, bringingthe total reported revision to**${formatSigned(totalNowcastRevision,2)} percentage points**. Only the firsttwo rows belong in the news sum Σ wᵢvᵢ.`
Try these comparisons.
Make the retail-sales innovation positive and its target weight negative. Check the sign of its contribution.
Set both innovations to zero while leaving the parameter revision nonzero. Separate the data-news revision from the total reported revision.
Hold the innovations fixed and change the weights. Explain why the same release can have different effects on different targets.
NoteWhy Good News Can Lower a Nowcast
The sign of a release’s contribution is the sign of w_iv_i. A positive innovation lowers the target when its model-implied target weight is negative. Negative weights can arise in a multivariate model after the covariance among the target, the release, and other observations is taken into account.
The weights in this lab stand in for that model-implied mapping; they are not chosen freely in a production decomposition. A zero innovation contributes nothing to the point revision, although observing the expected value can still reduce uncertainty. Parameter re-estimation is a separate source of change and should not be attributed to release news.
From One Release to a Real-Time GDP Nowcast
The five labs form one chain. A release enters a vintage on date r while remaining attached to reference period \tau. Its innovation and precision determine a state update. Repeated prediction and updating produce a filtered path, even when some observations are missing. The mixed-frequency measurement row maps monthly states into quarterly GDP, and the news decomposition reports the resulting target revision release by release.
Before leaving the page, explain that chain without using the formulas. If one step is unclear, return to the corresponding lab and change only the input responsible for that step. The matrix filter in the notes generalizes the same logic to many states and many observations.
The derivations, assumptions, and notation are in the Session 1 notes.
plotStyle = ({background:"transparent",color:"var(--bs-body-color)",fontFamily:"Archivo, system-ui, sans-serif"})functionformatSigned(value, digits =2) {if (Math.abs(value) <0.5*Math.pow(10,-digits)) {return (0).toFixed(digits); }return (value >0?"+":"−") +Math.abs(value).toFixed(digits);}functionmulberry32(a) {returnfunction () { a |=0; a = (a +0x6D2B79F5) |0;let t =Math.imul(a ^ (a >>>15),1| a); t = (t +Math.imul(t ^ (t >>>7),61| t)) ^ t;return ((t ^ (t >>>14)) >>>0) /4294967296; };}functiongaussian(rng) {let spare =null;returnfunction () {if (spare !==null) {const value = spare; spare =null;return value; }let u =0;while (u ===0) u =rng();const radius =Math.sqrt(-2*Math.log(u));const theta =2*Math.PI*rng(); spare = radius *Math.sin(theta);return radius *Math.cos(theta); };}