Skip to main content

Hyperparameter Random Search

This is the first hyperparameter search for the CNN-GRU hybrid neural network to forecast the energy load of residential areas using multiple features, incl. the load, sunshine minutes, dry temperature, daylengths, weekends, weekdays, and months.
Created on May 3|Last edited on May 3

Section 1 - Validation Loss

A parallel coordinate plot displaying the parameters used in the runs for the four lowest (<= 0.1444) validation losses is shown below. Further, the parameter importance is shown together with the correlation between the parameter and the lowest validation loss.

sgdoptimizer120130140150160170180190200210220230batch_size0.000.050.100.150.200.250.300.350.400.450.50dropout0.0240.0260.0280.0300.0320.0340.0360.0380.0400.0420.0440.0460.0480.050learning_rate0.9000.9050.9100.9150.9200.9250.9300.9350.9400.9450.950momentum500520540560580600620640660680700720lookback20406080100120140160180200220240cnn_layer_s...406080100120140160180200cnn_layer_s...406080100120140160180200220240gru_layer_s...406080100120140160180200220gru_layer_s...1.01.11.21.31.41.51.61.71.81.92.0num_cnn_layers1.01.11.21.31.41.51.61.71.81.92.0num_gru_layers0.0750.0800.0850.0900.0950.1000.1050.1100.115best_val_loss0.0850.0900.0950.1000.1050.1100.1150.1200.1250.1300.1350.140val_loss
Run set
4


Training and Validation Performance Metrics


Run set
4


Section 2 - Validation MAE + MAPE

This section provides an overview of the parameters used for achieving the lowest MAE and MAPE during the runs. Below four runs, identical to the four in section 1, are shown together with their respective parameter importances.

Run set
4


Section 3 - Which NOT to use

The three worst runs are shown below with their selected parameters.

Run set
12