Return to the STAT 248 Lab Homepage

Latex Macros

$ \newcommand{\E}[0]{\mathbb{E}}$ $ \newcommand{\P}[0]{\mathbb{P}}$ $ \newcommand{\V}[0]{\text{Var}}$ $ \newcommand{\Var}[0]{\V}$ $ \newcommand{\Cov}[0]{\text{Cov}}$ $ \newcommand{\Corr}[0]{\text{Corr}}$ $ \newcommand{\nor}[0]{\mathcal{N}}$ $ \newcommand{\a}[0]{\alpha}$ $ \newcommand{\b}[0]{\beta}$ $ \newcommand{\g}[0]{\gamma}$ $ \newcommand{\r}[0]{\rho}$ $ \newcommand{\p}[0]{\phi}$ $ \newcommand{\s}[0]{\sigma}$ $ \newcommand{\t}[0]{\theta}$ $ \newcommand{\T}[0]{\Theta}$ $ \newcommand{\Ph}[0]{\Phi}$ $ \newcommand{\diff}[0]{\nabla}$ $ \newcommand{\bg}[0]{\boldsymbol\gamma}$ $ \newcommand{\bp}[0]{\boldsymbol\phi}$ $ \newcommand{\br}[0]{\boldsymbol\rho}$ $ \newcommand{\Gam}[0]{\Gamma}$ $ \newcommand{\Z}[0]{\mathbb{Z}}$ $ \newcommand{\R}[0]{\mathbb{R}}$ $ \newcommand{\mand}[0]{\text{ and }}$ $ \newcommand{\ra}[0]{\rightarrow}$ $ \newcommand{\dist}[1]{\overset{#1}{\sim}} $

Introduction

  • Part 1 - SARIMA Models
  • Part 2 - Frequency Domain

Today I will cover the following topics:

  • Seasonal ARMA
  • Seasonal ARIMA

Relevant references include:

  • Sections 3.4 and 3.8 from Shumway and Stoffer
  • Chapters 10 from Cryer and Chan
In [1]:
library(TSA)
Loading required package: leaps
Loading required package: locfit
locfit 1.5-9.1 	 2013-03-22
Loading required package: mgcv
Loading required package: nlme
This is mgcv 1.8-15. For overview type 'help("mgcv-package")'.
Loading required package: tseries

Attaching package: ‘TSA’

The following objects are masked from ‘package:stats’:

    acf, arima

The following object is masked from ‘package:utils’:

    tar

In [2]:
data(co2)
plot(co2,ylab='CO2')
In [3]:
plot(window(co2,start=c(2000,1)),ylab='CO2')
Month=c('J','F','M','A','M','J','J','A','S','O','N','D')
points(window(co2,start=c(2000,1)),pch=Month)

Consider a series $$ Y_t = Z_t – \Theta Z_{t – 12} $$ Notice that $$ \Cov(\,Y_t, Y_{t – 1}\,) = \Cov(\, Z_t – \Theta Z_{t – 12},\, Z_{t – 1} – \Theta Z_{t – 13} \,) = 0 $$ but $$ \Cov(\,Y_t, Y_{t – 12}\,) = \Cov(\, Z_t – \Theta Z_{t – 12},\, Z_{t – 12} – \Theta Z_{t – 24} \,) = -\Theta \s^2 $$ The series is stationary and has non-zero autocorrelation at lag 12.

Define the seasonal MA(Q) model of order $Q$ with seasonal period $s$ by $$ Y_t = Z_t – \Theta_1 Z_{t – s} – \T_2 Z_{t – 2s} – \ldots – \Theta_Q Z_{t – Qs} $$ with seasonal MA characteristic polynomial $$ \Theta(x) = 1 – \Theta_1 x^s – \Theta_2 x^{2s} – \cdots - \Theta_Q x^{Qs} $$

For invertibilty, the roots of $\T(x)$ must all be greater than 1 in absolute value.

Analogously, define a seasonal AR($P$) model of order $P$ and seasonal period $s$ by $$ Y_t = \Ph_1 Y_{t – s} + \Ph_2 Y_{t – 2s} + \cdots + \Ph_P Y_{t – Ps} + Z_t $$ with seasonal characteristic polynomial $$ \Ph(x) = 1- \Ph_1 x^s - \Ph_2 x^{2s} - \cdots - \Ph_P x^{Ps} $$

Multiplicative Seasonal ARMA Models

Usually our models will incorporate autocorrelation at seasonal and neighboring slags.

Consider a model whose MA characteristic polynomial is given by $$ (1 - \t x) ( 1 - \T x^{12}) $$ The corresponding time series is $$ Y_t = Z_t - \t Z_{t-1} - \T Z_{t-12} + \t \T Z_{t-13} $$

Exercise: Verify that the autocorrelation function is nonzero only lags 1, 11, 12, and 13.

More generally, a multiplicative seasonal ARMA$(p,q)\times(P,Q)_s$ model with seasonal period $s$ is a model with AR characteristic polynomial $\p(x)\Ph(x)$ and MA characteristic polynomial $\t(x)\T(x)$, where $$ \begin{align*} \p(x) &= 1- \p_1 x - \p_2 x^2 - \cdots - \p_p x^p \\ \Ph(x) &= 1- \Ph_1 x^s - \Ph_2 x^{2s} - \cdots - \Ph_P x^{Ps} \\ \end{align*} $$ and $$ \begin{align*} \t(X) &= 1 - \t_1 x - \t_2 x^2 - \cdots - \t_q x^q \\ \T(x) &= 1 - \T_1 x^s - \T_2 x^{2s} - \cdots - \T_Q x^{Qs} \end{align*} $$

In other words, if $Y_t$ is ARMA$(p,q)\times(P,Q)_s$ then it satisfies the characteristic equation $$ \Ph(B) \phi(B) Y_t = \T(B) \t(B) Z_t $$

As always may contain a constant term $\alpha$. Is this simply an ARMA model with AR order $p + Ps$ and MA order $q + Qs$? Note that the coefficients are not that general. They are determined by $p + P + q + Q$ parameters as opposed to $p + Ps + q + Qs$ coefficients. With $s=12$, the former leads to a more parsimonious model.

Example: Suppose $P = q = 1$ and $p = Q = 0$ with $s = 12$. The model is then $$ Y_t = \Ph Y_{t-12} + Z_t + \t Z_{t-1} $$

Exercise: Use the lagged multiplication technique, show that $$ \g_1 = \Ph \g_{11} - \t \s^2 $$ and $$ \g_k = \Ph \g_{k-12} $$ Implying that $$ \begin{align*} \g_0 &= \frac{ 1 +\t^2}{1 - \Ph^2} \s^2 \\ \r_{12k} &= \Ph^k \quad \text{for } k \geq 1\\ \r_{12k-1} &= \r_{12k+1} =- \frac{\t}{1 + \t^2} \Ph^k \quad \text{for } k \geq 0 \\ \end{align*} $$

In [4]:
phi = c(rep(0,11),.8)
ACF = ARMAacf(ar=phi, ma=-.5, 50)[-1] # [-1] removes 0 lag
PACF = ARMAacf(ar=phi, ma=-.5, 50, pacf=TRUE)
par(mfrow=c(2,1))
plot(ACF, type="h", xlab="lag", ylim=c(-.4,.8)); abline(h=0)
plot(PACF, type="h", xlab="lag", ylim=c(-.4,.8)); abline(h=0)

Nonstationary Seasonal ARIMA

Recall the seasonal difference of period $s$ operator: $$ \diff_s Y_t = Y_t - Y_{t-s} $$

Consider the process $$ Y_t = S_t + Z_t $$ where $$ S_t = S_{t-s} + W_t $$ and $\{Z_t\}$, $\{W_t\}$ are independent white noise processes. $S_t$ is in some sense a seasonal random walk, and if $\s_W << \s_Z$ then $S_t$ is a slowly changing seasonal component.

Question: If a process has a seasonal component, is it stationary?

$Y_t$ is nonstationary since $S_t$ is. Applying the seasonal difference operator, we get that $$ \diff_s Y_t = S_t - S_{t-s} + Z_t - Z_{t-s} = W_t + Z_t - Z_{t-s} $$ which has the same autocorrelation as an MA$(1)_s$ process.

Consider a different model $$ Y_t = M_t + S_t + Z_t $$ where $$ S_t = S_{t-s} + W_t $$ and $$ M_t = M_{t-1} + U_t $$ where $Z_t, W_t, \mand U_t$ are mutually independent white noise processes.

As before we might start off with a seasonal difference $$ \diff_s Y_t = M_t - M_{t-s} + W_t + Z_t - Z_{t-s} $$ and then take its difference $$ \begin{align*} \diff \diff_s Y_t &= Mt - M{t-s} + W_t + Zt - Z{t-s}

  • (M{t-1} - M{t-s-1} + W{t-1} + Z{t-1} - Z_{t-s-1}) \ &= Ut - U{t-s} + Wt - W{t-1} + Zt - Z{t-1}
  • Z{t-s} + Z{t-s-1} \ & = (U_t + W_t + Zt) - (W{t-1} + Z{t-1} ) - (U{t-s} + Z{t-s} ) + Z{t-s-1} \end{align*} $$ A stationary process with nonzero autocorrelation at lags 1, $s-1$, $s$, and $s+1$. It agrees with the autocorrelation structure of an ARMA$(0,1)\times(0,1)_s$ model.

SARIMA Models

$Y_t$ is multiplicative seasonal ARIMA model (SARIMA) regular orders $p, d, \mand q$; seasonal orders $P, D, \mand Q$; and seasonal period $s$ if the differenced series $$ W_t = \diff^d \diff_s^D Y_t $$ satisfies an ARMA$(p,q) \times (P,Q)_s$. $Y_t$ is denoted as ARIMA$(p,d,q) \times (P,D,Q)_s$.

Note that this means that we can write the model as $$ \Ph(B) \phi(B) \diff_s^D \diff^d Y_t = \T(B) \t(B) Z_t $$

In [5]:
par(mfrow=c(3,1))
acf(as.vector(co2),lag.max=36)
plot(diff(co2),ylab='First Difference of CO2',xlab='Time')
acf(as.vector(diff(co2)),lag.max=36)
In [6]:
par(mfrow=c(2,1))
plot(diff(diff(co2),lag=12),xlab='Time',
ylab='First and Seasonal Difference of CO2')
acf(as.vector(diff(diff(co2),lag=12)),lag.max=36,ci.type='ma')
In [7]:
m1.co2 = arima(co2, order=c(0,1,1),
               seasonal=list(order=c(0,1,1),period=12))
m1.co2
Call:
arima(x = co2, order = c(0, 1, 1), seasonal = list(order = c(0, 1, 1), period = 12))

Coefficients:
          ma1     sma1
      -0.5792  -0.8206
s.e.   0.0791   0.1137

sigma^2 estimated as 0.5446:  log likelihood = -139.54,  aic = 283.08
In [8]:
par(mfrow = c(2,1))
plot(window(rstandard(m1.co2),start=c(1995,2)),
ylab='Standardized Residuals',type='o')
abline(h=0)
acf(as.vector(window(rstandard(m1.co2),start=c(1995,2))),
lag.max=36)
# hist, qqplot
In [9]:
?tsdiag
tsdiag(m1.co2)

The ARIMA$(0,1,1)\times(0,1,1)_{12}$ model was popularized in the first edition of the seminal book of Box and Jenkins (1976) when it was found to characterize the logarithms of a monthly airline passenger time series. This model has come to be known as the airline model.

In [10]:
library(astsa)
In [14]:
#?prodn
plot(prodn)
a = acf2(prodn, 48)
In [15]:
plot(diff(prodn))
b = acf2(diff(prodn), 48)