Lecture 12

Lecture 12

Outline

Time Series

Components of a Time Series

A time series consists of four components:

A time series may exhibit none of these components or maybe just one or two.

Trend Component

The overall pattern in the plot of the time series is called the trend component.

ts1

Seasonal Component

The seasonal component of a time series is the part of the variation that fluctuates in a way that is roughly stable over time with respect to timing, direction, and magnitude.

ts2

Cyclical Component

Regular cycles in the data with periods longer than one year are referred to as cyclical components.

When a cyclical component can be related to a predictable phenomenon, then it can be modeled based on some regular behavior and added to whatever model is being built for the time series.

Irregular Component

In time series modeling, the residuals – the part of the data not fit by the model – are call the irregular component.

Typically the variability of the irregular component is of interest – whether the variability changes over time or whether there are any outliers or spikes that may deserve special attention.

A time series that has a relatively constant variance is said to be stationary in the variance.

Example 1

Data from the U.S. Bureau of Labor gives Output/hr Labor and Output/unit Capital. Analyze the time series plot below.

ts3

Modeling Time Series

Methods for forecasting a time series fall into two general classes: smoothing methods and regression-based modeling methods.

Although the smoothing methods do not explicitly use the time series components, it is a good idea to keep them in mind. The regression models explicitly estimate the components as a basis for building models.

Smoothing Methods

Simple Moving Average Methods

\[\tilde{y_t} = \frac{\sum_{i=t-L+1}^t y_i}{L}\]

Example 2

Consider the plot of a time series for the closing price of a stock over the course of a year.

sma

The forecasted values for MA(5) (blue) and MA(15) (brown) are plotted below. Note that the longer moving average is smoother but reacts to rapid changes in the data more slowly.

Simple Moving Average Methods (cont.)

To obtain a forecast for a new time point, analysts use the last average in the series.

\[\hat{y}_{t-1} = \tilde{y}_{t}\]

This is the simple moving average forecast.

Moving averages are often used as summaries of how a time series is changing. Outliers tend to affect means and may distort the moving average summary.

Weighted Moving Averages

A weight can be assigned to each value in a weighted averaging scheme according to how far it is before the current value. Each value is multiplied by a weight before summing, and the total is divided by the sum of the weights.

\[\tilde{y}_t = \frac{\sum w_i y_{t-i}}{\sum w_i}\]

Two types of weighted moving average smoothers are commonly used on time series data: exponential smoothers and autoregressive moving averages.

Exponential Smoothing Methods

Exponential smoothing is a weighted moving average with weights that decline exponentially into the past.

\[\tilde{y}_t = \alpha y_t + (1-\alpha)\tilde{y}_{t-1}, 0< \alpha < 1\]

All previous values are used in exponential smoothing with distant values getting increasingly smaller weight. This can be seen by expanding the equation above to obtain the equation below.

\[\tilde{y}_t = \alpha y_t + \alpha(1-\alpha)\tilde{y}_{t-1} + \alpha(1-\alpha)^2\tilde{y}_{t-2} + \cdots\]

Example 3

Below we see a plot of the closing price of a stock (data shown in earlier slide) with exponentially smoothed values using $\alpha = 0.75$ (brown) and $\alpha = 0.10$ (green).

ema

Forecast Error

We define the forecast error at any time $t$ as:

\[e_t = y_t - \hat{y}_{t}\]

To consider the overall success of a model at forecasting for a time series we can use the mean squared error (MSE).

\[MSE = \frac{1}{n} \sum(y_t - \hat{y}_{t})^2\]

Forecast Error (cont.)

The MSE penalizes large errors because the errors are squared, and it is not in the same units as the data. We address these issues by defining the mean absolute deviation (MAD).

\[MAD = \frac{1}{n} \sum |y_t - \hat{y}_{t}|\]

Forecast Error (cont.)

The most common approach to measuring forecast error compares the absolute errors to the magnitude of the estimated quantity. This leads to what is called the mean absolute percentage error (MAPE).

\[MAPE = 100 \times \frac{1}{n} \sum \frac{|y_t - \hat{y}_{t}|}{|y_t|}\]

Autoregressive Models

Simple moving averages and exponential smoothing methods are good choices for series with no regular long-term patterns.

To find the weights, we can use the methods of multiple regression.

Autoregressive Models (cont.)

If we fit a regression to predict a time series from its lag1 and lag2 versions,

\[\hat{y} = b_o + b_1 y_{lag_1} + b_2 y_{lag_2}\]

ar2

Autoregressive Models (cont.)

The correlation between a series and a (lagged) version of the same series that is offset by a fixed number of time periods is called autocorrelation. The table shows some autocorrelations for the previous example.

ar2

Autoregressive Models (cont.)

A regression model that is based on an average of prior values in the series weighted according to a regression on lagged version of the series is called an autoregressive model.

\[\hat{y} = b_o + b_1 y_{lag_1}+ \cdots + b_p y_{lag_p}\]

Example 4

For the closing price of a stock data and its four lag variables in a previous slide, we find the coefficients for a fourth-order autoregressive model.

ar-ex

Because a fourth-order model is created, the model can be used to predict four time periods into the future.

Example 4 (cont.)

From the table in the previous slide, we obtain the following fourth-order autoregressive model.

\[y_t = 0.196 + 0.959 y_{lag_1} - 0.046 y_{lag_2} - 0.057y_{lag_3} + 0.133 y_{lag_4}\]

ar-ex2

Random Walks

The naïve forecast model is sometimes called a random walk because each new value can be thought of as a random step away from the previous value.

\[y_{t+1} = y_t + \varepsilon_t\]

where $\varepsilon_i$ are independent random values with some distribution.

Multiple Regression-based Models

The models studied so far do not attempt to model these components directly. Modeling the components directly can have two distinct advantages.

Modeling the Trend Component

When a time series has a linear trend, it is natural to model it with a linear regression of $y_t$ on Time.

Attractive feature of a regression-based model:

When the time series doesn’t have a linear trend, we can often improve the linearity of the relationship with a re-expression of the data.

Additive and Multiplicative Models

Adding dummy variables to the regression of a time series on Time turns what was a simple one-predictor regression into a multiple regression.

If we model the original values, we have added the seasonal component, S, (in the form of dummy variables) to the trend component, T, (in the form of an intercept coefficient and a regression with the Time variable as a predictor).

\[\hat{y}_t = T+S\]

This is an additive model because the components are added in the model.

Additive and Multiplicative Models (cont.)

After re-expressing a time series using the logarithm, we can still find a multiple regression. Because we are modeling the logarithm of the response variable, the model components are multiplied and we have a multiplicative model

\[\hat{y}_t = T \times S\]

Although the terms in a multiplicative model are multiplied, we always fit the multiplicative model by taking logarithms, changing the form to an additive model that can be fit by multiple regression.

Using Cyclical and Irregular Components

Time series models that are additive over their trend component, seasonal component, cyclical component, and irregular components may be written

\[\hat{y}_t = T+S+C+I\]

Time series models that are multiplicative over their trend component, seasonal component, cyclical component, and irregular components may be written

\[\hat{y}_t = T \times S \times C \times I\]

Forecasting with Regression-based Models

Regression models are easy to use for forecasting, and they can be used to forecast beyond the next time period. The uncertainty of the forecast grows the further we extrapolate.

Choosing a Time Series Forecasting Method

Simple moving averages demand the least data and can be applied to almost any time series. However:

Choosing a Time Series Forecasting Method (cont.)

Exponential smoothing methods have the advantage of controlling the relative importance of recent values relative to older ones. However:

Choosing a Time Series Forecasting Method (cont.)

Autoregressive moving average models use automatically determined weights to allow them to follow time series that have regular fluctuations. However:

Choosing a Time Series Forecasting Method (cont.)

Regression-based models can incorporate exogenous variables to help model business cycles and other phenomena. They can also be used to forecast into the future. However: