Regularization Techniques in Time Series
                                Yulia Gel
                          University of Waterloo
 

This talk addresses a "large p small n" problem in a time series
framework and considers different types of regularization of an empirical
information (covariance) matrix of a time series process.

In particular, the assumption that observed time series follows a model
with a finite number of parameters $p$ is rarely justified in practice.
One of the common approaches is to approximate the true unknown, possibly
non-linear, model by "long" autoregressive (AR) equations. Simple
structure along with the well established estimation techniques and
asymptotic properties for AR models make such an approximation very
attractive
for applications. Often approximation by ``long'' AR models is also used
as a benchmark against more sophisticated models. Usually the order $p$ of
an AR model is selected by information criterions (IC), and parameters are
then estimated by the Maximum Likelihood (ML), Least Squares (LS) or other
methods. However, in the real time cases  when sample size $T$ increases
and modelling and forecasting are performed online, it often implies
that the order of approximation $p$ should be refined and, thus, all $p$
parameters need to be recalculated. Hence, the computational cost of
approximation eventually increases. Such situations are widely met in a
variety of modern applications, e.g. prediction of stock or currency
exchange returns, filtering electrocardiogram measurements of heart
rate etc. Alternatively, we can utilize a ``smoothed'' version of the AR
approximation, which is based on regularizing the information (covariance)
matrix in the LS method, using a nuclear  ridge operator. Utilizing the
regularized information matrix enables to fit a much longer AR($p$) model
to the observed data than typically suggested by any IC, while recursively
estimating model parameters with different levels of accuracy, i.e.
the first AR coefficients are estimated more precisely than the tail
coefficients.  The regularizer controls how many parameters are to be
estimated precisely and the level of accuracy. Our results show that the
Regularized Least Squares (RLS) estimates converge almost surely even for
AR models of infinite order (AR($\infty$)) that can be viewed as a
limiting case of an AR approximation. In addition, we present the results
on regularization of an information matrix by banding and its impact on
parameter estimates and forecasts.

This work is joint with Peter Bickel, Andrey Barabanov and Bei Chen.