Apex DQN vs DQN

Our Apex DQN implementation reaches an episodic return of 340 in just 2 hours and 34 mins, which is even better than PPO's performance that reaches an average episodic return of in 3 hours.

• source code (single file implementation): https://github.com/vwxyzjn/cleanrl/blob/master/cleanrl/experiments/apex_dqn_atari_visual.py
• videos of agents playing games: https://app.wandb.ai/cleanrl/cleanrl.benchmark/runs/3f7o9vtj/files/BreakoutNoFrameskip-v4__apex_dqn_atari_visual__1__1597873510