Point Cloud Classification Using PyTorch Geometric

Table of Contents

The ModelNet Benchmarks

First, a list of the most common object categories in the world was compiled, using the statistics obtained from the SUN database.
Once a vocabulary for objects had been established, 3D CAD models belonging to each object category were collected using online search engines by querying for each object category term.
Then, human workers were manually used to decide whether each CAD model belongs to the specified categories using TurkCleaner, a design tool with quality control making use of Amazon Mechanical Turk service.

The ModelNet10 Dataset

The ModelNet40 Dataset

Diving Into the Workflow

Various methods for deep learning on graphs and other irregular structures from a variety of published papers.
A large number of common benchmark datasets (based on simple interfaces to create our own datasets).
Easy-to-use mini-batch loaders for operating on many small and single giant graphs and other irregular data structures such as molecules, 3D meshes, and point clouds.
Multi-GPU support.
Distributed graph learning via Quiver.
The GraphGym experiment manager for designing and evaluating Graph Neural Networks.

The PointNet++ Architecture

The grouping phase constructs a graph in which nearby points are connected. Typically, this is either done via k-nearest neighbor search or via ball queries (which connect all points that are within a radius to the query point).
The neighborhood aggregation phase executes a Graph Neural Network layer that, for each point, aggregates information from its direct neighbors (given by the graph constructed in the previous phase). This allows PointNet++ to capture local context at different scales.
The downsampling phase implements a pooling scheme suitable for point clouds with potentially different sizes. We will ignore this phase for now and will come back later to it.

The Hierarchical Feature Learning Architecture, used by PointNet++. Source: Figure 2 from the paper.

The Grouping Phase

The Sampling layer selects a set of points from input points, which defines the centroids of local regions.
The Grouping layer then constructs local region sets by finding “neighboring” points around the centroids.
The PointNet layer uses a mini-PointNet to encode local region patterns into feature vectors.

[Bonus Section] - Learn How PyG implements the PointNet Layer using Message Passing Networks - [Click to Expand] 👉

Implementing the Set Abstraction Layer

The Classification Head

The architecture of our PointNet++ classification model. Source: Figure 2 from the paper.

class PointNetPlusPlus(torch.nn.Module):
    def __init__(
        self,
        set_abstraction_ratio_1, set_abstraction_ratio_2,
        set_abstraction_radius_1, set_abstraction_radius_2, dropout
    ):
        super().__init__
        # Input channels account for both `pos` and node features.
        self.sa1_module = SetAbstraction(set_abstraction_ratio_1, set_abstraction_radius_1, MLP([3, 64, 64, 128]))
        self.sa2_module = SetAbstraction(set_abstraction_ratio_2, set_abstraction_radius_2, MLP([128 + 3, 128, 128, 256])
        self.sa3_module = GlobalSetAbstraction(MLP([256 + 3, 256, 512, 1024]))
        self.mlp = MLP([1024, 512, 256, 10], dropout=dropout, norm=None)

    def forward(self, data):
        sa0_out = (data.x, data.pos, data.batch)
        sa1_out = self.sa1_module(*sa0_out)
        sa2_out = self.sa2_module(*sa1_out)
        sa3_out = self.sa3_module(*sa2_out)
        x, pos, batch = sa3_out
        return self.mlp(x).log_softmax(dim=-1)



Training the Point-Cloud Classifier

Experiments

Tuning the Hyperparameters

It is very evident that the dropout rate has a very strong negative correlation with our metric. This indicates that we can lower the dropout rate without worrying much about our model overfitting.
The number of points being sampled per point cloud in the input pipeline has a strong positive correlation with our metric. This is evident given that a higher number of points indicate a point cloud that is denser and of higher resolution.

Final Experiment

We would go with a very low dropout rate.
We would select a high sample number which basically means a higher resolution of point clouds.
Due to a lack of clarity with a clear trend in batch size, we would go with a batch size of 16.
We would take the rest of the hyperparameters from the best-performing run in the sweep.

Conclusion

In this report, we explore how to build a model for classifying 3D point clouds.
We explored how we could easily build data loaders for point cloud classification using PyTorch Geometric's dataset API.
We also explored the PointNet++ architecture for point cloud classification as proposed by the paper PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space.
We explore how we can implement the PointNet++ architecture using the building blocks provided to us by PyTorch Geometric.
We briefly explored the concept of Message Passing Networks and how to implement a Message Passing layer using PyTorch Geometric.
We trained our model on the ModelNet10 dataset and showcased how to use Weights & Biases to track our experiment and store and version our checkpoints with a PyTorch-based training and validation loop.
We used Weights & Biases Sweeps to correctly identify the optimal set of hyperparameters and train the final version of our model.
The implementation of PointNet++ in this report is inspired by this example on PyTorch Geometric repository.
You can check out more similar reports: