Report
This report was created to optimize my network for text generation and also bring more understanding to RNN's in general and how to optimize them.
Github repository: https://github.com/borisd13/char-RNN
Created on January 30|Last edited on February 5
Comment
RNN, LSTM or GRU?
Those runs were performed to compare different type of recurrent cells: "vanilla" RNN, LSTM, GRU.
LSTM & GRU clearly outperform RNN.
GRU seems also to learn faster than LSTM so it could be the preference.
It is important to also observe the impact of time. By changing the absissa to time (vs Step or Epoch), we can see that RNN is much faster than LSTM & GRU, but unfortunately stops learning much earlier. However, I was also running several experiments at the same time which clearly affected the run time of each individual one.
Run set 1
3
Run set 1
3
Run set 1
4
Run set 1
4
Run set 1
5
Run set 1
4
Run set 1
4
Run set 1
6
Add a comment