Let’s begin with the ARCH model.
ARCH Model
The ARCH model was initially raised by Engle (1982), and the ARCH model means the Autoregressive Conditional Heteroskedasticity model.
We assume here \(u_t\) is the return.
$$ u_t=\frac{P_t-P_{t-1}}{P_{t-1}} $$
$$u_t\sim N(0,\sigma_t^2)$$
The data-generating process (DGP) is like an AR form, as the name of ARCH. The volatility is autoregressively generated by \(u^2_i\).
$$\sigma_t^2=\delta_0+\sum_{i=1}^{p} \delta_i u_{t-i}^2$$
, where \(p\) is the number of lags, and \(\delta_i\) are a set of parameters. The DGP of that model shows that the volatility of the return is heteroscedastic, correlated with the squared term of the return per se.
For example, an ARCH(1) model is like,
$$ \sigma_t^2=\delta_0+\delta_1 u^2_{t-1} $$
- Stationarity
Note here we need our time series to be stationary for better forecasting. Thus, \(Var(u_t)=\sigma^2 \)
$$ Var(u_t)=\delta_0+\delta_1 Var(u_{t-1}) $$
$$ \sigma^2=\frac{\delta_0}{1-\delta_1} $$
As the variance has to be positive. We need \(\delta_0 > 0\), and \(\delta_1<1\).
- Estimation
For this time series data, OLS assumptions are violated, because our series are autoregressive heteroskedasticity.
Instead, the Maximum Likelihood Estimation (MLE) would be a better estimation method by assuming the probability distribution of variables.
MLE allows iterations to find parameters \(\delta\) that can maximise the maximum likelihood function.
GARCH Model
The ‘G’ in the GARCH model means ‘generalised’, and the GARCH model has a set of additional terms, \(\sum \gamma_i \sigma^2_i \). Thus, the DGP of the GARCH(p,q) model is as the following,
$$u_t\sim N(0,\sigma_t^2)$$
$$ \sigma_t^2=\delta_0 + \sum_{i=1}^{p} \delta_i u^2_{t-i} +\sum^q_{j=1} \gamma_j \sigma^2_{t-j} $$
ARMA-GARCH Model
That is a further application, in which the GARCH model is applied to mimic the movement of error terms in the ARMA model.
We initially assume an ARMA(p,q) model,
$$ y_t=\beta_0 +\sum^p_{i=1} \beta_i y_{t-i} + \sum^{q}_{j=1} \theta_j u_{t-j} +u_t$$
Then, we assume the error term here, \(u_t \sim GARCH(m,n)\).
$$ u_t \sim N(0,\sigma_t^2)$$
$$ \sigma_t^2 = \delta_0 +\sum^m_{i=1} \delta_i u_{t-i}^2 +\sum_{j=1}^n \gamma_n \sigma_{t-n}^2 $$
Reference
Engle, R.F., 1982. Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation. Econometrica: Journal of the econometric society, pp.987-1007.