Maybe it's just all about model size.
I think the models I've been using is indeed a bit too small.
Created on February 28|Last edited on March 2
Comment
The difficulty_id task is quite hard because of the weak label from Henle, and originally I thought the ASAP dataset with Henle labeling can only achieve around 50-60% accuracy, with graph being slightly better. After I upgraded the cnn model to resnet (which is at least 100x larger than the graph and original), there is a significant improvement.
Parameter counts:
Resnet8: 4.7M
Resnet18: 11.7M
Graph (GAT and hetero): ~0.1M
Old CNN I was using: 0.03M
But after the model gets to a certain scale more parameters doesn't help anymore. Resnet18 is no better than Resnet8 (which is a personal modification I did since I really don't think we need this scale of model. Even 18 is too much. (but in the existing works they are using 50.)) Resnet18 seems to overshoot more easily, but I should probably go with a smaller learning rate.
Does increasing the GNN up to the scale of resnet helps? I also tried with a 6.7M parameters GNN, with 8 layers and the hidden size incremented to 256 (GAT). That doesn't seems to be the proper way of scaling GNNs. Will write another report in scaling gnn.
Section 1
Run set
0
Add a comment