Pouring, SVD Methods (Ablation Test)

This is more of an ablation where we compare pointwise vs MSE losses.
Created on June 4|Last edited on June 5
Comment
﻿
(06/03/2022) Ran the following ablation on the proposed method, with MSE instead of pointwise loss (but still using consistency loss):
﻿
# Same as `SVD_POINTWISE_6D_EE2FLOW_SVD` except that we will just take the
# (R,t) produced from the flow, turn it to a 6D vector, and do MSE on that.
# TL;DR test MSE loss instead of pointwise tool flow loss. This will involve
# doing basically the same "flow2act" that we do at inference time, except
# it is done at TRAIN time, and so the forward pass just returns a 6D vector.
SVD_6D_FLOW_EEPOSE_MSE_LOSS = dict(
    obs_type='point_cloud',
    act_type='eepose',
    encoder_type='pointnet_svd_6d_flow_mse_loss',
    method_flow2act='svd',  # this should be a duplicate of the forward pass
    use_consistency_loss=True,  # actually this makes sense to use
    lambda_consistency=0.1,
    scale_targets=True,
    lambda_pos=1.0,
    lambda_rot=1.0,  # 100 for MixedMedia, 1 for PourWater?
)
Also compare with: https://wandb.ai/mooey5775/mixed_media/reports/Pouring-SVD-Methods--VmlldzoyMTAzNDU1﻿
Compare with the proposed method:
# 6D flow, but using SVD to extract actions, learning from >= 4DoF demos.
# As of 06/01/2022 this is serving as our 'proposed' method, WITH the
# consistency loss and WITHOUT the separate MLPs for (R,t).
SVD_POINTWISE_6D_EE2FLOW_SVD = dict(
    obs_type='point_cloud',
    act_type='ee2flow',
    encoder_type='pointnet_svd_pointwise_6d_flow',
    method_flow2act='svd',
    use_consistency_loss=True,
    lambda_consistency=0.1,
    scale_pcl_flow=True,
    scale_pcl_val=250,
    separate_MLPs_R_t=False,
)
(06/05/2022) Interestingly, it seems like the pointwise loss actually helps here! This is good news (and does make sense from a "principled" perspective) though the performance gap isn't too large (though I'm hoping that is expected).
Results, Ablation, MSE vs Pointwise (6D Flow)﻿
SVD_6D_FLOW_EEPOSE_MSE_LOSS (w/consistency)5
SVD_POINTWISE_6D_EE2FLOW_SVD (w/consistency)5
﻿
GIFs, of the MSE loss ablation(The pointwise one has been shown elsewhere.) Hmm, seems somewhat similar. No physics issues that could be causing things, etc.
﻿
﻿
﻿
﻿
﻿
﻿
Add a comment