BC #02: MMOneSphere, ntrain100, PNet2_svd_pointwise_flow

This is now doing SVD pointwise on the 4DOF demonstrator! So we actually have to use flow here, not `ee2flow` because `ee2flow` is only if we can copy the ee delta transitions equally to all the flow vectors, and we can't do that with >3DOFs. Unfortunately, the results here aren't that good ... so we're going to have to brainstorm ways to improve on this (or maybe adjust the demonstrator, how the policy produces and implements actions, etc.).

Daniel Seita

Created on April 29|Last edited on April 29

Comment

Success rates are not that good. Argh, compare that with the ee2flow which is almost directly comparable except translation only and using ee2flow means we get more info that's lost due to collisions.
The GIFs look interesting, there is actually some rotation being predicted, but interestingly in the wrong direction, do we have the axes set correctly? Pretty sure we do, but could that pose issues during training? Also maybe a good demonstrator shouldn't be doing a lot of rotation in 1 axes, but a little bit of rotation in all 3 axes, I wonder if that is in some way easier to learn? 
Finally, consider that we use axis-angle formulation during training. That means the network has to predict a negative value for rotating one way, and a positive value for rotating another way, and it entirely depends on the value of one component, as we clip the x and z portions. Is this a concern?
Success Rates and Train/Eval MSEsRemember here that train/MSE_loss is based on the FLOW vectors, so it's 3D for each flow, then we average across the minibatch (only with the TOOL points, I confirmed again due to my paranoia...). That's why there is no MSE_loss_rot here (it's just set to 0, we don't "optimize" directly on a rotation but on the predicted vs ground truth flow vectors, where predicted flow is produced from the given rotation).
﻿
Run set3
﻿
Example GIFsFirst seed after 250 epochs:
﻿
Second seed after 250 epochs:
﻿
Third seed after 250 epochs:
﻿
VariantNote the action_type: svd and translation_axis_angle which should override the initial env_kwargs['action_mode'] and it has to given that we do see some rotations.
{
  "_hidden_keys": [],
  "act_type": "flow",
  "action_type": "svd",
  "actor_lr": 0.0001,
  "agent": "bc",
  "alg_policy": "ladle_algorithmic_v04",
  "algorithm": "BC",
  "batch_size": 24,
  "bc_data_dir": "/data/dseita/softgym_mm/data_demo/MMOneSphere_v01_BClone_filtered_ladle_algorithmic_v04_nVars_2000_obs_combo_act_translation_axis_angle",
  "bc_data_filtered": true,
  "data_buffer_capacity": 1000000,
  "encoder_type": "pointnet_svd_pointwise",
  "env_kwargs": {
    "action_mode": "translation",
    "action_repeat": 8,
    "camera_name": "top_down",
    "deterministic": false,
    "headless": true,
    "horizon": 100,
    "num_variations": 1000,
    "observation_mode": "cam_rgb",
    "render": true,
    "render_mode": "fluid"
  },
  "env_kwargs_action_mode": "translation_axis_angle",
  "env_kwargs_camera_height": 128,
  "env_kwargs_camera_width": 128,
  "env_kwargs_deterministic": false,
  "env_kwargs_num_variations": 2000,
  "env_kwargs_observation_mode": "point_cloud",
  "env_name": "MMOneSphere",
  "env_version": "v01",
  "exp_name": "MMOneSphere_v01_BC_ntrain_0100_PCL_PNet2_svd_pointwise_flow_2022_04_28_15_40_09_0001",
  "hidden_dim": 1024,
  "lambda_pos": 1.0,
  "lambda_rot": 100.0,
  "log_interval": 1,
  "n_epochs": 250,
  "n_eval_episodes": 10,
  "n_train_demos": 100,
  "num_filters": 32,
  "num_layers": 4,
  "project_axis_ang_y": true,
  "save_freq": 10,
  "save_model": true,
  "save_video": true,
  "seed": 100,
  "wandb_entity": "mooey5775",
  "wandb_project": "mixed_media",
  "weighted_MSE": false
}
﻿

Add a comment