About the model

I am using a trajectory prediction model* that incorporates features from the environment in the form of an obstacle grid. In all shown results the models are pretrained using the zara03 dataset. This dataset doesn't include an obstacle grid, therefore only the parameters of the relevent part of the model are warmstarted. Furthermore the model uses an autoencoder to extract features from the grid. The autoencoder is also pretrained using the mnist dataset.

As stated by Pfeiffer et. al. "Importantly, in order to be applicable to a widerange of situations and environments, overfitting to a certain environment needs to be avoided." . This is exactly the problem that will be tackled in this report.

*Pfeiffer, M., Paolo, G., Sommer, H., Nieto, J., Siegwart, R., & Cadena, C. (2018, May). A data-driven model for interaction-aware pedestrian motion prediction in object cluttered environments. In 2018 IEEE International Conference on Robotics and Automation (ICRA) (pp. 1-8). IEEE.

Scenario One: Cooperative hallway

Scenario Two: Obstacle hallway

Scenario Three: Square hallway loop

Discussion

As expected for every scenario the corresponding model performs best. However, what I want to highlight here is the generalization performance or lack thereof. This is especially significant for the model trained in the square scenario. The model overfits the scenario. While the model will still perform well on validation/test data of the scenario, the model will completely fail to generalize to different situations.

Online data aggregation approach

The 'Combined' model uses an aggregated dataset containing all three scenarios. In all three scenarios the combined model performs well and thus seems to generalize better. This is to be expected ofcourse because it was trained on data from all scenarios. Nevertheless, an autonomous robot could theoretically aggregate data from the environments it encounters while deployed. Thus, we show that aggregating the dataset and subsequently training on the aggregated dataset improves the generalization across all.

Online gradient descent

Yet the approach of aggregation data from every single scenario we will ever encounter is not feasible. At some point the aggregated dataset will simply become too large to store or to train on. Instead, an online gradient descent approach could provide a solution. In an Online gradient descent approach, every timestep a single gradient descent step is taken and thus the memory requirements are very low.

Follow up research

To test if an online gradient descent approach can train a model that generalizes across situations, I have run a model subsequently in all scenarios. In each scenario, the model was trained for 10 minutes using OGD. It turns out that the order of the scenarios influences the performance a lot. When the model was trained on the square scenario last, it again completely overfit to the square scenario and significantly lost performance on the other scenarios. This loss of performance can be attributed to the model overwriting it's previously learned weights. Thus the model 'catastrophically forgets' what it has previously learned when the previously learned behavior is not experienced for a while.

results

I ran the model in the following order: cooperative → obstacle → square. As the bar charts below illustrate, the model completely overwrites its previously learned behavior and overfits to the square scenario.

Mitigating catastrophic forgetting

Continual learning

In the sub-field of continual learning, catastrophic forgetting is one of the main problems. There are many proposed methods to mitigate it with the aim of containing as much information about the previous task/scenario/data distribution as possible. In an influential paper that came out three years ago* a regularization strategy was introduced to mitigate catastrophic forgetting in neural networks called "Elastic weight biasing". Additionally maintaining more information about past examples can help. An easy way of doing so is by maintaining a fixed-size buffer of examples. By making sure the buffer size is constrained, the method is still computationally tractable in the long run.

*Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., ... & Hassabis, D. (2017). Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13), 3521-3526.