Only ToolFlowNet and Direct Vector MSE, for both variants of PourWater (3D and 6D action spaces). Uses PyFlex, which is what we can release!

Only ToolFlowNet and Direct Vector MSE (an older version), for both variants of PourWater (3D and 6D action spaces). Uses PyFlexRobotics, NOT PyFlex.

Now we have RGBD input (woo hoo) and can test them. This report has them for all three versions (pouring, and 4d and 6d scooping). Note: RGBD for here, not RGB (unless strictly for comparison purposes). UPDATE: actually now I feel like Depth-Segmentation will give better comparisons.

Lots of results here! :)

To serve as a baseline.

See ToolFlowNet (and other methods) with and without this fix.

For both 3D and 6D action spaces, for both scooping tasks. For both keypoints and images (the latter means CURL). Also keypoints == state information.

How do these compare?

4DoF demo data (again, trying now). See other report for 3DoF demo data.

With single and multi-sphere cases.

This a summary of the training results for ToolFlowNet, when trained on the newer algorithmic (translation-only) demonstrator.

After CoRL, trying more data augmentation using the 50 demo case.

Post-CoRL. 50 demo case.

This should be for our 'proposed' 3D and 6D flow methods; ablations can go elsewhere.

Now with updated data statistics so we can track how often we are getting one set of balls over another, etc. (Edit: investigating again in mid-July but maybe make a new report as it's harder to group runs, for some reason.)

Trying to get the flow methods to work the best on translation-only data for scooping. So far we have something that shows some promise. However, it still feels unreliable and a reviewer who is watching the videos may not be impressed.

Might as well combine these together? Not sure we have many things to talk about here.

Experiment with adding noise to point cloud.

Now we just have 5 such points in the point cloud! (Edit: now 10)

This is an ablation for 3D flow, to be clear, not 6D flow in case we are using 3D flow as the method.

Focusing on training of baseline methods here. Just do translation data.

Finally moving towards 4D manipulation. NOTE: will also put some ablations here. This will focus on 6D flow, to be clear.

Focusing on training of baseline methods here. Just do translation data.

Check (a) the new experimental setting, (b) scaling / no scaling variants.

(1) No skip connections, (2) pointwise before SVD, (3) using dense loss

(1) No skip connections, (2) pointwise before SVD.

How does performance vary as a function of demonstration count?

Experiment with adding noise to point cloud. See here for visuals on what extra noise means to the point clouds, etc.

Should not work as good as the single-sphere case.

Why is this baseline so awful? We really have to investigate, I think.

At last we are trying some SVD methods with pouring.

This is more of an ablation where we compare pointwise vs MSE losses.

Let's try the dense transformations methods. It seems like this will be giving slightly better performance relative to pouring?

These are baseline methods that we hope do not do well. :). UPDATE: also adding the naive PN++ baseline but where we use pointwise loss! That will tell us if pointwise is sufficient, or if we need more. (UPDATE 06/02/2022, running again, see notes here)

Check to see how much performance varies based on # of demos.

Now we're going to deal with pouring as our second environment. This has a bunch of the 'naive' baselines.

This is the simple dense transformation we've been talking about.

Using 3D flow. Testing with and without consistency. Objective: compare 3D flow with 6D flow and see if 3D is worse (due to coupling between the rotation and translation).

Classification PN++. Testing with and without the older data.

Has 15 trials with and without the consistency loss (and, w/out consistency but with OLDER v02 3DoF data). After the 05/16 bug fix where we properly scale `data.pos`.

For different methods, compare the effect of # of training data points.

Trying out the simplest Dense Transform possible. NOTE: this is using 3DoF dense transform so it is translation only.

Using older data (ladle_algorithmic_v02) to debug

Seems to be doing pretty well.

This is the 'naive' way of using 'classification' PN++ to regress directly to the 3D position delta, but it seems to do a good job here despite some strange validation MSEs (sometimes they are spiking upwards). [Edit: that's because of dropout...]

Naive method, however I expect it to work well on the translation only case. I'm testing with and without data augmentation, the data augm uses randomized image crops.

At test time, we just keep the first 3 dimensions of flow (which are translations) and run the average of it.

The average is NON differentiable, just something we do after the fact. Also unfortunately we can't look at the evaluation flow plots because those were not wrapped in the `utils.eval_mode()` and thus have the dropout making it look weird. :( Might want to re-do these, then.

Using older data (ladle_algorithmic_v02) to debug.

3DoF demonstrators, testing pointwise loss using `flow` instead of `ee2flow`, which might be more realistic but may lose information related to collisions.

WITHOUT the consistency loss. NOTE: cannot look at the evaluation flow visualizations or MSEs because that doesn't have the correct model.eval() but the action selection should be the same. Unfortunately doesn't look that good?

WITH the consistency loss. NOTE: cannot look at the evaluation flow visualizations or MSEs because that doesn't have the correct model.eval() but the action selection should be the same. Unfortunately doesn't look that good?

Action repeat=1, very slow.

More tests with overfitting to 1 demonstration, now with (fixed?) flow and rotation. Some promising results and interesting observations, mostly with 4DoF demonstrators.

Need to see if this can overfit to 1 demonstration. Done before running larger-scale BC#03 experiments, with 10 runs total (5 for 1-sphere, 5 for multi-sphere).

Hopefully flow does better. Actually it seems like it does? See the comparisons here

See if flow can be used with SVD for the multi-sphere case.

Averaging layer, which was doing the best with 1 sphere env.

Naive method with classification PN++, regressing to 3DoF pose, now for multi-sphere env. Update: also testing with the RGB network.

On the newer dataset now regressing straight to `eepose` so it's 6DoF axis angle, and then for test time we end up clearing out the x and z axis for the rotation so it's just the y axis. Looks bad which is good (well, assuming our other methods do better). NOTE: this hack could be an issue because the delta (amount of rotation) depends on the magnitude of the axis-angles. Of course, again the MSEs all had x and z as 0 so if it predicts something there it isn't learning correctly...

Now checking SVD pointwise on the 4DoF demonstrator! Need to also carefully check the demonstrator since I had some bugs in my earlier code with rotations.

I actually have 7 results here, 1 from Eddie, 3X runs from me 04/27, and 3X more runs 04/29 (should be the same, I was debugging something but I think all 7 are under same experimental settings). These show pointwise SVD, but we are still using a 3DoF action space. This means the output of the rotation is ignored (see notes below).

Try the naive averaging layer this time using the new BC dataset. Remember, this was the one that did really well, but only for the translation-only data, so we hope to do at least as good as this with SVD if we have no rotations in the demonstrator. UPDATE: yes, results are really good! Now if only we can get SVD to do as well as this. :X

This report will analyze the "naive" way of BC with 3DOF data / actions: we take the segmented point cloud, pass it as input to a classification PN++, and then do MSE straight on the 3DoF translation. I also used this to debug / investigate MSEs. I also test with 100 and 500 training demos.

This is now doing SVD pointwise on the 4DOF demonstrator! So we actually have to use flow here, not `ee2flow` because `ee2flow` is only if we can copy the ee delta transitions equally to all the flow vectors, and we can't do that with >3DOFs. Unfortunately, the results here aren't that good ... so we're going to have to brainstorm ways to improve on this (or maybe adjust the demonstrator, how the policy produces and implements actions, etc.).

Now this is just classification PointNet++, instead of regressing straight to EE delta, we regress to the flow which has been averaged to the EE delta. This is OK since there are no rotations here. But now this has a lot of clipped values so I expect performance to be a bit lower compared to the `BC_PCL_PNet2_ee` case.

This is the raw flow, so we are: (a) doing MSE per-point separately on all tool points, (b) get the 'action' from the agent by averaging the output. Not working that well, results seem consistent with what I've seen previously.

Here we are now backpropagating through an averaging layer, so this is using segmentation PN++ but regressing to a single EE via an average at the end (just 3D position change). Surprisingly this seems to work very well. In fact these learning curves look REALY good.

Try the backprop-through-average. Compare this with `BC_PCL_PNet2_avg_ee` the only difference is whether we use `flow2ee` (here) or the raw ee, which reflects the effect of collisions. Since we use flow, I am guessing performance should be a bit worse. (Edit: maybe not! It looks good here!) Remember, here we have 3D flow per point, and since demonstrator just did translation, that turns into our "EE position change" as averaging all the flow means we get the same thing.

Segmentation version of PN++, regressing onto flow. Challenges: (a) MSE is per-point, (b) flow loses collision info. So this is not expected to work well, and is mainly a baseline. (Update: as expected, pretty bad performance)

Now we're running through the differentiable SVD layer for the first time. We'll use EE as targets, and the MSE is applied on both the translation and the quaternion, though the quaternions are all the identity here. The quaternion part should enforce some consistency with the flow predictions, right? (Update: results do show some promise! But they need to be better...)

Multi-sphere env with 1 distractor. This is the "naive" way where we have a classification variant of PN++ and we just regress to the target 3D EE position change.

Let's see if flow2ee is a bit worse compared to just ee. This is with the classification PN++. It would be good to get more verification that the action magnitudes could cause a difference in performance (this seems reasonable to me).

Segmentation based PN++ where it outputs flow and where we do point-wise MSE against the ground-truth flow. This is likely not to be that good even if we use "ee" information for the flow. Update: yes it's pretty bad.

Now doing the multi-sphere env with 1 distractor (might be hard). Here we have the segmentation PN++ with an averaging layer so we backprop through the average.

Now running through differentiable SVD layer with the multi-sphere env. Again this is translation-only so the quaternions are kind of irrelevant but we do try and apply a MSE on the quaternions to make the resulting flow map to (1,0,0,0).

To compare with: (a) PN++ with classification flow2ee, so to see effect of the two architectures of PN++, (b) PN++ with average layer but with ee, to see the effect (deterioration?) of using flow instead of ee with the loss of collision information.