Demo Report
This report was created to optimize my network for text generation and also bring more understanding to RNN's in general and how to optimize them.
Github repository: https://github.com/borisd13/char-RNN
Created on February 13|Last edited on February 13
Comment
RNN, LSTM or GRU?
Those runs were performed to compare different type of recurrent cells: "vanilla" RNN, LSTM, GRU.
LSTM & GRU clearly outperform RNN.
GRU seems also to learn faster than LSTM so it could be the preference.
It is important to also observe the impact of time. By changing the absissa to time (vs Step or Epoch), we can see that RNN is much faster than LSTM & GRU, but unfortunately stops learning much earlier. However, I was also running several experiments at the same time which clearly affected the run time of each individual one.
Run set 1
3
Run set 1
3
Run set 1
4
Run set 1
4
Run set 1
5
Run set 1
4
Run set 1
4
Run set 1
6
Add a comment