VAR for multivariate/vector-valued time series

A standard Vector Autoregression (VAR) for vector-valued time series is \[ \mathbf{y}_t = \sum_{i=1}^p \mathbf{A}_i \mathbf{y}_{t-i} + \mathbf{e}_t, \] with \(\mathbf{A}_i \in \mathbb{R}^{N \times N}\) and \(\mathbf{y}_t \in \mathbb{R}^{N}\)

Common Features

• “A feature will be said to be common if a linear combination of the series fails to have the feature even though each of the series individually has the feature” Engle and Kozicki (1993)
• Focus on common serial correlation (SCCF) for multiple time series: \[ \mathbf{y}_t = \sum_{i=1}^p \textcolor{red}{\mathbf{B}} \mathbf{C}_i'\mathbf{y}_{t-i} + \mathbf{e}_t, \quad \mathbf{A}_i = \textcolor{red}{\mathbf{B}} \mathbf{C}_i' \]

How do we interpret this?

• Suppose we observe the GDP of three countries: United States (USA), Canada (CAN), and Great Britain (GBR)
• We estimate the SCCF model with \(r = 1\) and, after rotation, find \[ \mathbf{B}_\perp' = \begin{bmatrix} 1 & 0 & -2 \\ 0 & 1 & -1 \\ \end{bmatrix} \]

Index Models

\[ \mathbf{y}_t = \sum_{i=1}^p \mathbf{B}_i \textcolor{red}{\mathbf{C}'}\mathbf{y}_{t-i} + \mathbf{e}_t, \quad \mathbf{A}_i = \mathbf{B}_i \textcolor{red}{\mathbf{C}'} \]

Matrix Autoregressive Model (MAR)

For matrix-valued time series, we have the MAR(p) model \[ \mathbf{Y}_t = \sum_{i=1}^p \mathcal{A}_i \bar{\times}_2 \mathbf{Y}_{t-i} + \mathbf{E}_t, \] where \(\mathbf{Y}_t \in \mathbb{R}^{N_1 \times N_2}\), \(\mathcal{A}_i \in \mathbb{R}^{N_1 \times N_2 \times N_1 \times N_2}\), and \(\bar{\times}_2\) is the contraction along the second dimension

• Contraction is the generalization of matrix multiplication extended to tensors
• In our example, \(N_1\) are economic indicators and \(N_2\) are countries

Contraction Details

Low Rank Structure

\[ \mathbf{Y}_t = \sum_{i=1}^p \mathcal{A}_i \bar{\times}_2 \mathbf{Y}_{t-i} + \mathbf{E}_t \]

We assume \(\mathcal{A}_i\) has a Tucker decomposition (Tucker 1966): \[ \mathcal{A}_i = \mathcal{G}_i \times_1 \mathbf{U}_{1} \times_2 \mathbf{U}_{2} \times_3 \mathbf{U}_{3} \times_4 \mathbf{U}_{4} \in \mathbb{R}^{N_1 \times N_2 \times N_1 \times N_2}, \] \[ \mathbf{U}_{1} \in \mathbb{R}^{N_1 \times r_1}, \quad \mathbf{U}_{2} \in \mathbb{R}^{N_2 \times r_2}, \] \[ \mathbf{U}_{3} \in \mathbb{R}^{N_1 \times r_3}, \quad \mathbf{U}_{4} \in \mathbb{R}^{N_2 \times r_4}, \] \[ \mathcal{G}_i \in \mathbb{R}^{r_1 \times r_2 \times r_3 \times r_4} \]

• \(\mathbf{U}_j, j = 1, \dots, 4\) may have different ranks, and \(\mathcal{G}_i\) ties it all together

Tucker Decomposition

• Visualizations are limited to three-dimensional tensors (rather than our four dimensional coefficient)

• \(\mathcal{A}_i\) is decomposed into a “compressed” core tensor \(\mathcal{G}_i\) and factor matrices \(\mathbf{U}_j\) for \(j = 1, \dots, 4\)

Equivalent VAR representation

• The Reduced Rank MAR(p) has an equivalent vectorized form, where \(\text{vec}(\mathbf{Y}_t) = \mathbf{y}_t \in \mathbb{R}^{N_1 N_2}\) and \(\text{mat}(\mathcal{A}_i) = \mathbf{A}_i \in \mathbb{R}^{N_1 N_2 \times N_1 N_2}\) \[ \mathbf{y}_t = \sum_{i=1}^p \underbrace{(\mathbf{U}_{2} \otimes \mathbf{U}_{1}) \mathbf{G}_{i} (\mathbf{U}_{4} \otimes \mathbf{U}_{3})'}_{\mathbf{A}_i} \mathbf{y}_{t-i} + \mathbf{e}_t \]

• \(\text{rank}(\mathbf{A}_i) = \min (r_1 r_2, r_3 r_4)\)

• Allows us to explore both SCCF and index model restrictions

Link to SCCF

• Our null space matrix from before (\(\mathbf{A}_\perp\)) now has a Kronecker structure \[ \mathbf{y}_t = \sum_{i=1}^p \textcolor{red}{(\mathbf{U}_{2} \otimes \mathbf{U}_{1})} \mathbf{G}_{i} (\mathbf{U}_{4} \otimes \mathbf{U}_{3})' \mathbf{y}_{t-i} + \mathbf{e}_t \] \[ (\mathbf{U}_{2} \otimes \mathbf{U}_{1})_\perp' \mathbf{y}_t = (\mathbf{U}_{2} \otimes \mathbf{U}_{1})_\perp' \mathbf{e}_t \]

Another Example

• We now observe the GDP and Industrial Production (PROD) of each country, with the same three countries as before: USA, CAN, and GBR
• Forms an \(N_1 \times N_2 = 2 \times 3\) matrix-valued time series
• All countries co-move, and all economic indicators co-move, resulting in rank \((r_1 = 1,r_2 = 1)\) for the SCCF model \[ \mathbf{U}_{1\perp}' = \begin{bmatrix} 1 & -1 \end{bmatrix}, \quad \mathbf{U}_{2\perp}' = \begin{bmatrix} 1 & 0 & -2 \\ 0 & 1 & -1 \\ \end{bmatrix} \] \[ y_{t, gdp} - y_{t, prod} = e_{t}^* \] \[ y_{t, usa} - 2 y_{t, gbr} = e_{t}^* \quad \text{and} \quad y_{t, can} - y_{t, gbr} = e_{t}^* \]
• We can separate the effects from the countries and the indicators

Connection with VAR coefficient

• Left illustrates a partial reduced rank of the indicators \(r_1 = 3\) and \(r_2 = N_2 = 5\)
• Right illustrates a partial reduced rank of the countries \(r_1 = N_1 = 4\) and \(r_2 = 4\)

Index/Factor Models

\[ \mathbf{y}_t = \sum_{i=1}^p (\mathbf{U}_{2} \otimes \mathbf{U}_{1}) \mathbf{G}_{i} \textcolor{red}{(\mathbf{U}_{4} \otimes \mathbf{U}_{3})'} \mathbf{y}_{t-i} + \mathbf{e}_t \]

Can be rearranged as

Tucker Rank Selection

• Standard information criteria (AIC and BIC) to jointly estimate the ranks and lags

\[ \small \text{AIC}(r_1, r_2, r_3, r_4, p) = \ln(|\widehat{\Sigma}_{\mathbf{E}}|) + \frac{2}{T} \phi(r_1, r_2, r_3, r_4, p), \] \[ \small \text{BIC}(r_1, r_2, r_3, r_4, p) = \ln(|\widehat{\Sigma}_{\mathbf{E}}|) + \frac{\ln(T)}{T} \phi(r_1, r_2, r_3, r_4, p), \] where \[ \small \phi(r_1, r_2, r_3, r_4, p) = \prod_{i=1}^{4} r_i p + \sum_{i=1}^2 (r_i (N_i - r_i) + r_{2+i} (N_i - r_{2+i})), \] is the number of parameters in the model

Simulation Study

• Selecting Tucker ranks with AIC and BIC

Fully reduced

True rank of \(\mathbf{r} = (1,1,1,1)\)

Partially reduced first dimension

True rank of \(\mathbf{r} = (3,1,3,1)\)

Partially reduced second dimension

True rank of \(\mathbf{r} = (1,4,1,4)\)

No rank reduction

True rank of \(\mathbf{r} = (3,4,3,4)\)

Macroeconomic indicators for various countries

Serial Correlation Common Features

• \((r_1=3,r_2=1)\) implies a partially reduced economic indicator dimension, while the country dimension is fully reduced

Economic Indicators

• Rank of \(r_1=3\): one co-movement relation between all four indicators
• Normalizing over the second variables (GDP), we obtain

• GDP movements are approximately three times that of manufacturing production across all countries, while CPI inversely co-moves with GDP

Serial Correlation Common Features

Countries

• Rank of \(r_2=1\): all countries co-move contemporaneously
• If we are interested in the co-movements of the U.S., we normalize the null space matrix accordingly

• Almost one-to-one co-movements of all countries with the U.S., with the Canadian economic indicators most closely moving together with the U.S. ones, as evident from the first column

Factor Models

• \((r_3=4,r_4=3)\) implies a full rank indicator dimension, while the country dimension is partially reduced
• We construct predictor factors \(\mathbf{F}_t^{pred} = \mathbf{U}_3^\top \mathbf{Y}_{t-1} \mathbf{U}_4 \in \mathbb{R}^{4 \times 3}\)

Conclusion

• RR-MAR captures dimension-specific co-movement dynamics in matrix time series at aggregate levels
• Uncovers contemporaneous and lagged relationships for fully/partially reduced rank structures
• Co-movements identified for all countries and strongest for GDP/PROD

Matricization/Unfolding

For any \(K\)-dimensional tensor \(\mathcal{Y} \in \mathbb{R}^{N_1 \times \dots \times N_K}\), denote its mode-\(k\) unfolding \(\mathbf{Y}_{(k)} \in \mathbb{R}^{N_k \times N_{-k}}\), with \(N_{-k}:= \prod_{i = 1, i \neq k}^K N_i\), as the matrix obtained by setting the \(k\)-th tensor mode as its rows and collapsing all the other dimensions into the columns. Specifically, the \((n_1, \dots, n_K)\)-th element of \(\mathcal{Y}\) is mapped to the \((n_k, j)\)-element of \(\mathbf{Y}_{(k)}\), where

\[ j = 1 + \sum_{\substack{l = 1,\\l \neq k}}^K (n_l - 1) J_l \quad \text{with} \quad J_l = \prod_{\substack{m = 1,\\m \neq k}}^{l - 1} N_m. \]

Examples

Let the frontal slices of a tensor \(\mathcal{X} \in \mathbb{R}^{3 \times 4 \times 2}\) be as follows \[ \scriptsize \begin{align*} \mathcal{X} = \left[ \begin{bmatrix} 1 & 4 & 7 & 10 \\ 2 & 5 & 8 & 11 \\ 3 & 6 & 9 & 12 \end{bmatrix}, \begin{bmatrix} 13 & 16 & 19 & 22 \\ 14 & 17 & 20 & 23 \\ 15 & 18 & 21 & 24 \end{bmatrix}\right] \end{align*} \] The three mode-\(n\) unfoldings are: \[ \scriptsize \begin{gather*} \mathbf{X}_{(1)} = \begin{bmatrix} 1 & 4 & 7 & 10 & 13 & 16 & 19 & 22 \\ 2 & 5 & 8 & 11 & 14 & 17 & 20 & 23 \\ 3 & 6 & 9 & 12 & 15 & 18 & 21 & 24 \end{bmatrix} \quad \mathbf{X}_{(2)} = \begin{bmatrix} 1 & 2 & 3 & 13 & 14 & 15 \\ 4 & 5 & 6 & 16 & 17 & 18 \\ 7 & 8 & 9 & 19 & 20 & 21 \\ 10 & 11 & 12 & 22 & 23 & 24 \end{bmatrix} \\ \mathbf{X}_{(3)} = \begin{bmatrix} 1 & 2 & 3 & 4 & 5 & \dots & 9 & 10 & 11 & 12 \\ 13 & 14 & 15 & 16 & 17 & \dots & 21 & 22 & 23 & 24 \end{bmatrix} \end{gather*} \] Back to main

Contraction

The mode-n product between a tensor \(\mathcal{X} \in \mathbb{R}^{I_1 \times I_2 \times \dots \times I_N}\) and a matrix \(\mathbf{U} \in \mathbb{R}^{J \times I_n}\) is then denoted \[ \small \begin{align*} (\mathcal{X} \times_{n} \mathbf{U})_{i_1, \dots, i_{n-1}, j, i_{n+1}, \dots, i_N} = \sum_{i_n = 1}^{I_n} x_{i_1, i_2, \dots, i_N} u_{j, i_n} \in \mathbb{R}^{I_1 \times \dots \times I_{n-1} \times J \times I_{n+1} \times \dots \times I_N} \end{align*} \]

This can also be expressed in terms of unfolded tensors as \[ \begin{equation*} \small \mathcal{Y} = \mathcal{X} \times_{n} \mathbf{U} \quad \Leftrightarrow \quad \mathbf{Y}_{(n)} = \mathbf{U} \mathbf{X}_{(n)} \end{equation*} \]

Contraction

Suppose we have \(\mathcal{X} \in \mathbb{R}^{I_1 \times \dots \times I_M}\) and \(\mathcal{Y} \in \mathbb{R}^{J_1 \times \dots \times J_N}\) with \(I_m = J_1\). The sequence of contracted products is denoted by \(\mathcal{X} \bar{\times}_{M} \mathcal{Y}\) and yields a \((M+N-2)\) order tensor \(\mathcal{Z} \in \mathbb{R}^{I_1 \times \dots \times I_{M-1} \times J_1 \times \dots \times J_{N-1}}\) with entries \[ \small \begin{align*} \mathcal{Z}_{i_1, \dots, I_{M-1}, j_2, \dots, j_N} = (\mathcal{X} \times_{M} \mathcal{Y})_{i_1, \dots, i_{M-1}, j_2, \dots, j_N} = \sum_{i_M = 1}^{I_M} \mathcal{X}_{i_1, \dots, i_{M-1}, i_M} \mathcal{Y}_{i_M, j_2, \dots, j_N} \end{align*} \] If we have a \(5 \times 4 \times 3 \times 2\) tensor \(\mathcal{A}\) and a \(3 \times 2\) matrix \(\mathbf{B}\), the sequence of contractions yields the matrix \[ \small \mathcal{A} \bar{\times}_2 \mathbf{B} = \mathbf{C} \in \mathbb{R}^{5 \times 4} \]

Back to main

Fully reduced

True rank of \(\mathbf{r} = (1,1,1,1)\)

Partially reduced first dimension

True rank of \(\mathbf{r} = (1,9,1,9)\)

Partially reduced second dimension

True rank of \(\mathbf{r} = (2,1,2,1)\)

No rank reduction

True rank of \(\mathbf{r} = (2,9,2,9)\)

Estimation by Least Squares

\[ \small \begin{flalign*} \widehat{\mathcal{A}} &= [[\widehat{\mathcal{G}}; \widehat{\mathbf{U}}_1, \widehat{\mathbf{U}}_2, \widehat{\mathbf{U}}_3, \widehat{\mathbf{U}}_4]] = && \end{flalign*} \] \[ \small \underset{\substack{\mathcal{G} \in \mathbb{R}^{r_1 \times r_2 \times r_3 \times r_4},\\ \mathbf{U}_{i} \in \mathbb{R}^{N_j \times r_i}}}{\arg\min} \left\{ \frac{1}{2T}\sum_{t=1}^T \|\text{vec}(\mathbf{Y}_t) - \left(\mathbf{U}_2 \otimes \mathbf{U}_1\right) \mathbf{G}_{[2]} \left( \mathbf{I}_p \otimes \mathbf{U}_4 \otimes \mathbf{U}_3\right)^\top \text{vec}(\mathcal{X}_{t})\|_F^2\right\}, \]

• Rearrange the lags into a separate dimension to make \(\mathcal{X}_t\)
• Non-convex optimization problem as with many reduced rank problems
• Use an alternating gradient descent scheme to solve

Projection Matrices

• Projection matrices \(\mathbf{U}_i \mathbf{U}_i'\) are exactly identified
• Tell us which variables are most important for each factor

Coincident and Leading Indexes for U.S. States

• Consider \(N_1 = 2\) indicators (coincident and leading) over \(N_2 = 9\) states: Iowa (IA), Illinois (IL), Indiana (IN), Michigan (MI), Minnesota (MN), North Dakota (ND), Ohio(OH), South Dakota (SD), and Wisconsin (WI)
• Coincident and Leading Indicators may not move with one another, but different states may co-move

Coincident and Leading Indexes for U.S. States

• AIC selects full rank with two lags, while BIC selects full rank with one lag
• Results suggest no evidence for co-movements among the indicators or the states
• Highlight the potential for identifying no rank reduction in the data