1. Introduction

Methods for two sets of forecasts, using hourly and daily profiles respectively, are discussed here. Profiles are modelled independently for now.

Why not Multivariate Time Series

It is known that there is no relationship between profiles from different households, because their behaviours are not directly correlated. The only regressor considered should be time. Others like temperature may contribute, but are unknown for now. So it is not necessary to use vector auto regression (VAR).

Why Facebook Prophet

Prophet is an open-source package for modelling and forecasting univariate time series by Facebook. Its model structure is similar to that of generalised additive models (GAM), where distinct non-linear terms are integrated in a linear way. Prophet works best when the series shows strong and multiple seasonality, and it is robust to missing values and effect from holidays, making it the perfect choice in our case.

2. Demo: Daily Profile of Household 0

Though profiles in the same cluster tend to show similar patterns, it is relatively easy to build different statistical model for different households. Daily profiles of household 0 is modelled by Facebook Prophet and seasonal ARIMA in this section.

Facebook Prophet

Point forecasts and confidence intervals are plotted.

Forcasts and confidence intervals for future 3 days:

yhat yhat_lower yhat_upper
date
2017-12-01 0.135803 0.085666 0.183257
2017-12-02 0.159741 0.110511 0.206479
2017-12-03 0.154350 0.108852 0.201318

According to the previous figure, there is a trending and a weekly seasonal components, which are shown in following two figures respectively. The trending component increased from 0.104 to 0.141 in 122 days. That is, the average power consumption increased by 34.93 % in merely 4 months. Such trending effect must be taken into account.

Surprisingly, there are variations in power consumption over one week. Household 0 tends to consume more during weekends and Thursday. The difference between values on Saturday and on Wednesday amounts to 28.98 % of the mean power consumption, so this effect is essential as well.

Residual analysis for this Prophet model has not been finished. Some of the techniques are demonstrated in specification tests and autocorrelation tests in section 3.

Seasonal ARIMA

It is impossible to plot ACF and PACF for original series because of missing entries. As discussed before, they are adjacent so it is hard to fill with meaningful values. Previous result provides a starting point. To have a weekly season is a good idea, and it should be "integrated", in order to remove the long-term trend.

For example, an integrated SARIMA (with AR1, MA1 and weekly seasonal AR1) can be built. The seasonal AR1 term (value 0.1603, std error 0.145, pvalue 0.269) is a bit controversial, but stays due to the previous finding. The normality and heteroskedasticity assumptions are both rejected, so this is not good model.

Good news is that there is nothing left according to ACF and PACF. This example is actually the final model selected after a manual model selection.

Pseudo Out-of-Sample Validation

It is challenging to conduct a cross validation for time series models, especially when there is long-term trend. Pseudo out-of-sample validation is used instead. Here is a description from statsmodels:

"A common use case is to cross-validate forecasting methods by performing h-step-ahead forecasts recursively using the following process"

  • Fit model parameters on a training sample
  • Produce h-step-ahead forecasts from the end of that sample
  • Compare forecasts against test dataset to compute error rate
  • Expand the sample to include the next observation, and repeat

MSE of 3-step forecast using Facebook Prophet is 0.00356, and its root is 48.83 % of mean power consumption. So this is not a good model.

MSE of 3-step forecast using seasonal ARIMA is 0.00370, which is 3.94 % higher than MSE using Prophet, and its root is 49.78 % of mean power consumption. Despite of the increment, this model structure will be used to model all other daily profiles. Two approaches are essential the same. It is easier to interpret seasonal ARIMA, but it cannot be used for hourly profiles, which is discussed in the following section.

3. Demo: Hourly Profile of Household 0

Prophet can even be used to model time series with hourly (sub-daily) resolution.

Here is a line plot for in-sample and out-of-sample forecasts and confidence intervals.

Out-of-sample forecasts for future 4 hours are:

yhat yhat_lower yhat_upper
datetime
2017-12-01 00:00:00 0.157450 0.026090 0.290846
2017-12-01 01:00:00 0.135131 -0.005102 0.282784
2017-12-01 02:00:00 0.115922 -0.021668 0.253831
2017-12-01 03:00:00 0.108154 -0.024035 0.245774

Multiple Seasonality

Here are plots for one trending and two seasonality components.

Unfortunately, it is impossible to apply seasonal ARIMA, because of the multiple seasonality.

Specification Tests

The assumption that residuals from Prophet model are normally distributed is rejected. Furthermore, according to White's test for Heteroscedasticity, variance of residuals does vary with power consumptions.

When power consumption is high, the in-sample forcast is expected to have a large deviance. The same should go for out-of-sample forecasts.

In spite of the previous finding, when residuals are compared to corresponding values, they can be 4 times larger. So it is hard to say that more precise forecasts can be made when power consumption is low (usually during nights).

Autocorrelation Tests

All the Ljung-Box tests for lags 1, 2, 3, 4, 5, 24, and 168 reject the assumption that the residual is not auto-correlated. The test result is supported by ACF, when missing values are dropped.

Seasonal ARIMA for Residuals

Such information can be modelled by a SARIMA with AR1, AR1 (24), MA1 (24). Then, independence assumptions in lags 1, 2, 3, 4, and 5 are not rejected. So, the model has been improved, which is supported by the following ACF.

However, the model is still not satisfying, because it does not pass normality test and heteroskedasticity test.

Pseudo out-of-sample validations for Prophet and that with additional seasonal ARIMA have not been finished. So Prophet is used to make 4-step forecast.

4. Future Work

  • To combine multiple series belonging to different units is not a good idea, through there is a simple way to do it (estimating same model over multiple time series, crossvalidated). As suggested before, there are trending components. Though they can be offset, such information is lost.
  • Apply more flexible methods and tune parameters using pseudo out-of-sample validation.
  • Generalized estimating equations (GEE) can be a next-stage model, because clustering results can be taken into consideration, though to apply GEE to time series is not easy. There are more training data when a general model for profiles in a whole cluster is to be built. Besides, scaling and translation variances should be removed first.
  • The fact that profiles are non-stationary because of weekends and holidays is challenging. It might help if same profile can be divided into two batches. Because the batch for such non-business-days has less observations, those from different households in the same cluster can be combined into one training set.

5. Results

Two sets of forecasts for 500 households are made. Note that the physical unit is presumed to be kiloWatt.