Skip to main content

BC03 Scooping 3D Flow Ablation: MSE after SVD (not pointwise)

This is an ablation for 3D flow, to be clear, not 6D flow in case we are using 3D flow as the method.
Created on June 12|Last edited on June 14
Results don't seem that good?
To be clear this is SVD_3D_FLOW_EEPOSE_MSE_LOSS and we can compare that with the 6D flow version of that ablation https://wandb.ai/mooey5775/mixed_media/reports/BC03-4DoF-1-Sphere-PN-SVD-Pointwise-6D-flow-ee2flow-100-demos--VmlldzoyMDM5ODgy and here:

# Same as `SVD_POINTWISE_6D_EE2FLOW_SVD` except that we will just take the
# (R,t) produced from the flow, turn it to a 6D vector, and do MSE on that.
# TL;DR test MSE loss instead of pointwise tool flow loss. This will involve
# doing basically the same "flow2act" that we do at inference time, except
# it is done at TRAIN time, and so the forward pass just returns a 6D vector.
SVD_6D_FLOW_EEPOSE_MSE_LOSS = dict(
obs_type='point_cloud',
act_type='eepose',
encoder_type='pointnet_svd_6d_flow_mse_loss',
method_flow2act='svd', # this should be a duplicate of the forward pass
use_consistency_loss=True, # actually this makes sense to use
lambda_consistency=0.1,
scale_targets=True,
lambda_pos=1.0,
lambda_rot=100.0, # 100 for MixedMedia, 1 for PourWater?
)

# Same as `SVD_POINTWISE_SVD` except that we will just take the (R,t) from
# the flow, turn it to a 6D vector, and do MSE on that. MSE instead of PW.
SVD_3D_FLOW_EEPOSE_MSE_LOSS = dict(
obs_type='point_cloud',
act_type='eepose',
encoder_type='pointnet_svd', # just keep this name
method_flow2act='svd', # this should be a duplicate of the forward pass
use_consistency_loss=True, # actually this makes sense to use
lambda_consistency=0.1,
scale_targets=True,
lambda_pos=1.0,
lambda_rot=100.0, # 100 for MixedMedia, 1 for PourWater?
)

NOTE: no training visualizations, I couldn't figure out how to get it working and was in somewaht of a rush to do these. I actually put these in a separate branch https://github.com/Xingyu-Lin/softagent_rpad/commit/9a8b25b0a11fda9695e7039d4fb18f40230491a8

Why are the train MSEs not well behaved? This is really strange, I am convinced there is something strange. :(
I thought there was a bug so I re-ran 5 of these on 06/13/2022 after: https://github.com/Xingyu-Lin/softagent_rpad/commit/5268472aae2de68382d34a6969ff9192386c4ff9
Still, that should have just avoided a duplicate function but maybe there is something else going on?
Update: yeah, results just don't seem that good.

Results


MSE after SVD Ablation (1st attempt)
5
MSE after SVD Ablation (2nd attempt)
5