Researchers Propose New Feature Extraction Method for Computer Vision
Researchers from Northeastern and Adobe uncover a new method for feature extraction
Created on March 8|Last edited on March 8
Comment
Convnets and Vision Transformers have been all the rage in computer vision over the last few years, but a few researchers from Northeastern University and Adobe have just created a new way to extract features from an image for later use in downstream tasks. They call their method Context Clusters (CoCs), and the method views the image as a set of unorganized points where features can be extracted via a simplified clustering algorithm.
Each pixel in the image is considered a 5-dimensional data point containing information about color and position, and this data is clustered to be used as a visual representation.
How it works
Context clustering begins by grouping points into clusters based on similarity. After creating the clusters, features are aggregated in each cluster based on similarities to the center point of the cluster.
These features are then adaptively dispatched to each point in a cluster based on similarity, which allows the points to communicate with one another and share features from all points in a cluster. Overall the authors create an architecture that involved multiple CoC blocks, similar to how a convolutional block would be used.
SOTA Performance with Interpretability
The authors note that this architecture has many benefits around interpretability as the clustering process can be easily visualized. In addition to these interpretability benefits, the authors were able to match the performance of many current SOTA models on tasks like image segmentation and detection on the MS-COCO dataset.
Fortune Favors the Brave
It’s exciting to see more experimentation in new methods for computer vision tasks, with risk-taking by researchers outside of the typical methodologies. Although it’s important to continue to improve existing methods, it’s essential that researchers continue to experiment with completely new methods in order to continue to elevate AI capabilities.
It’s likely that we will see much more work around CoC’s in the future, and their implementation as standalone architecture, as well as being combined with existing architectures should raise the SOTA in many tasks.
The Paper:
Add a comment
Tags: ML News
Iterate on AI agents and models faster. Try Weights & Biases today.