BC #02: MMOneSphere, ntrain100, PNet2_svd_pointwise_ee2flow

I actually have 7 results here, 1 from Eddie, 3X runs from me 04/27, and 3X more runs 04/29 (should be the same, I was debugging something but I think all 7 are under same experimental settings). These show pointwise SVD, but we are still using a 3DoF action space. This means the output of the rotation is ignored (see notes below).
Daniel Seita
Created on April 28|Last edited on May 2
Comment
How exactly does this version work?
This uses ee2flow so it will take ee (the 3D translation deltas) and duplicate those to produce flow (only works because we have 3DoF demonstrator and action spaces).
The PN++ will output flow, and then we call flow2pose to get the (R,t) transformation.
Then we apply that on our data to get a predicted flow, i.e., given the current observation point cloud, we apply (R,t) on it to transform it. That gives us two point clouds, before and after, and this directly gives us 3D flow.
For training, we do MSEs on that plus ground truth 3D flow.
For test time steps, take predicted flow, and convert it to a translation (and rotation) but with using the tip correction.
IN THIS CASE because we have 3DoF (not in file name, we might want to change that but it's in the variant.json) we are just going to dump the rotation part (was originally quaternion, then I changed to axis angle, but that doesn't matter here) and just run the translation deltas, i.e., env.step(action[:3]).
Good news: this is producing better results (given same # of demos) compared to the naive way: https://wandb.ai/mooey5775/mixed_media/reports/BC-02-MMOneSphere-Naive-Method-Regress-Straight-to-3DoF--VmlldzoxOTEzNTUz﻿
BUT ... it seems like it is still not as good as a simple averaging layer?? For the averaging layer you can find the report here:
﻿https://wandb.ai/mooey5775/mixed_media/reports/BC-02-MMOneSphere-ntrain100-PNet2_avg_ee--VmlldzoxOTE0MTk3﻿
From looking at the binary success rate, it does seem to be following an expected pattern of getting high success rates and then declining a bit (due to overfitting). But the averaging layer seems to be a lot more reliable...
Combing all the 7 runs, we get:
0.3956 +/- 0.29 ... hmm, this is actually a bit worse than what we see with the flow variant which actually loses some information. Is this just due to noise in training? Still, results do show some promise and it is better than the naive method.
Success Rates and Train/Eval MSEs.﻿
Eddie (04/27)1
Daniel (04/27)3
Daniel (04/29)3
Combine all runs7
﻿
Example GIFs of runs 04/27:
Example GIFs of runs 04/29:(In our code this is using encoder_type = pointnet_svd_pointwise as of April 30.)
1st seed after 250, hmm ... I wonder if we need to change the demonstrator so that it actually lifts as soon as it sees the item?
﻿
2nd seed after 250, not as much sideways movement as I would have liked...
﻿
3rd seed after 250, looking OK especially when considering that it pushed the ball inside the water
﻿
These have the bug fix:
﻿https://github.com/Xingyu-Lin/softagent_rpad/commit/c2c9443401f9ec101e5fa89709c27fcfdfb6684f﻿
and also:
﻿https://github.com/mooey5775/softgym_MM/commit/6d137139c89c7a3fe451849fa732a91040c832f8﻿
Variant of one of the seeds:
(base) dseita@seuss:~/softagent_rpad_MM/data/local$ cat MMOneSphere_v01_BC_ntrain_0100_PCL_PNet2_svd_pointwise_ee2flow/MMOneSphere_v01_BC_ntrain_0100_PCL_PNet2_svd_pointwise_ee2flow_2022_04_29_22_32_30_0001/variant.json 
{
  "_hidden_keys": [],
  "act_type": "ee2flow",
  "action_type": "svd",
  "actor_lr": 0.0001,
  "agent": "bc",
  "alg_policy": "ladle_algorithmic_v02",
  "algorithm": "BC",
  "batch_size": 24,
  "bc_data_dir": "/data/dseita/softgym_mm/data_demo/MMOneSphere_v01_BClone_filtered_ladle_algorithmic_v02_nVars_2000_obs_combo_act_translation",
  "bc_data_filtered": true,
  "data_buffer_capacity": 1000000,
  "encoder_type": "pointnet_svd_pointwise",
  "env_kwargs": {
    "action_mode": "translation",
    "action_repeat": 8,
    "camera_name": "top_down",
    "deterministic": false,
    "headless": true,
    "horizon": 100,
    "num_variations": 1000,
    "observation_mode": "cam_rgb",
    "render": true,
    "render_mode": "fluid"
  },
  "env_kwargs_action_mode": "translation",
  "env_kwargs_camera_height": 128,
  "env_kwargs_camera_width": 128,
  "env_kwargs_deterministic": false,
  "env_kwargs_num_variations": 2000,
  "env_kwargs_observation_mode": "point_cloud",
  "env_name": "MMOneSphere",
  "env_version": "v01",
  "exp_name": "MMOneSphere_v01_BC_ntrain_0100_PCL_PNet2_svd_pointwise_ee2flow_2022_04_29_22_32_30_0001",
  "hidden_dim": 1024,
  "lambda_pos": 1.0,
  "lambda_rot": 100.0,
  "log_interval": 1,
  "n_epochs": 250,
  "n_eval_episodes": 10,
  "n_train_demos": 100,
  "num_filters": 32,
  "num_layers": 4,
  "project_axis_ang_y": false,
  "save_freq": 10,
  "save_model": true,
  "save_video": true,
  "seed": 100,
  "wandb_entity": "mooey5775",
  "wandb_project": "mixed_media",
  "weighted_MSE": false
}
﻿
Add a comment