Prediction Intervals on Test Set: Forecasting with Fable

[evphil88]

Hello,
In the online text books (2nd and 3rd editions) there is a section devoted to the test and train approach to determining model accuracy. I am following the 3rd edition and would like to extract confidence intervals on the from the one-step forecast done on the test set. I am unable to share data because this is for work. However I am able to obtain the fitted values with the code below

fit%>%####ie the model fitted on the train data
refit(test)%>%###ie apply the model to the test data or one step forecast on the test data
fitted()###give the fitted values

The code above returns a tsible object with .model, Date, and .fitted parameters.

But shouldn't there be prediction intervals as well?
Am I totally off here and there are no prediction intervals for 1-step forecasts?

Thank You!

---

^{Referred here by Forecasting: Principles and Practice, by Rob J Hyndman and George Athanasopoulos}

[robjhyndman]

The one step forecast variance is simply the residual variance, so it is easy to construct prediction intervals if you need them. Here is an example where 95% prediction intervals are computed assuming Gaussian residuals.

library(fpp3)
#> ── Attaching packages ──────────────────────────────────────────── fpp3 0.4.0 ──
#> ✓ tibble      3.1.3      ✓ tsibble     1.0.1 
#> ✓ dplyr       1.0.7      ✓ tsibbledata 0.3.0 
#> ✓ tidyr       1.1.3      ✓ feasts      0.2.2 
#> ✓ lubridate   1.7.10     ✓ fable       0.3.1 
#> ✓ ggplot2     3.3.5
#> ── Conflicts ───────────────────────────────────────────────── fpp3_conflicts ──
#> x lubridate::date()    masks base::date()
#> x dplyr::filter()      masks stats::filter()
#> x tsibble::intersect() masks base::intersect()
#> x tsibble::interval()  masks lubridate::interval()
#> x dplyr::lag()         masks stats::lag()
#> x tsibble::setdiff()   masks base::setdiff()
#> x tsibble::union()     masks base::union()

train <- aus_production %>%
  filter(Quarter < yearquarter("2006 Q1"))
test <- aus_production %>%
  filter(Quarter >= yearquarter("2006 Q1"))

fit <- train %>%
  model(ets = ARIMA(Beer))
sigma2 <- glance(fit) %>% pull(sigma2)
fit %>%
  refit(test) %>%
  fitted() %>%
  mutate(
    lo = .fitted - 1.96*sqrt(sigma2),
    hi = .fitted + 1.96*sqrt(sigma2)
  )
#> # A tsibble: 18 x 5 [1Q]
#> # Key:       .model [1]
#>    .model Quarter .fitted    lo    hi
#>    <chr>    <qtr>   <dbl> <dbl> <dbl>
#>  1 ets    2006 Q1    438.  407.  469.
#>  2 ets    2006 Q2    386.  355.  417.
#>  3 ets    2006 Q3    405.  374.  436.
#>  4 ets    2006 Q4    491.  460.  522.
#>  5 ets    2007 Q1    428.  397.  459.
#>  6 ets    2007 Q2    379.  347.  410.
#>  7 ets    2007 Q3    396.  365.  427.
#>  8 ets    2007 Q4    482.  451.  513.
#>  9 ets    2008 Q1    420.  389.  451.
#> 10 ets    2008 Q2    371.  340.  402.
#> 11 ets    2008 Q3    387.  356.  418.
#> 12 ets    2008 Q4    475.  444.  506.
#> 13 ets    2009 Q1    427.  396.  458.
#> 14 ets    2009 Q2    391.  360.  422.
#> 15 ets    2009 Q3    405.  374.  436.
#> 16 ets    2009 Q4    486.  455.  518.
#> 17 ets    2010 Q1    430.  399.  461.
#> 18 ets    2010 Q2    397.  366.  428.

^{Created on 2021-08-07 by the reprex package (v2.0.0)}

[evphil88]

Ah, of course! Thank you Dr. Hyndman!