An Introduction to Convolutional Graph Neural Networks

This article provides a beginner-friendly introduction to Convolutional Graph Neural Networks (GCNs), which apply deep learning paradigms to graphical data.
Saurav Maheshkar
Created on March 3|Last edited on March 3
Comment
﻿
﻿
Convolutional networks have dominated the vision world for quite a long time in deep learning. Admittedly attention is becoming the new de facto standard in vision however, there is another domain in which convolution has been making strides recently, graphical data !! 
In this article we'll introduce the notion of Graph Convolutional Networks (GCNs), a type of Spatial Network like Message Passing Graph Networks. 
What are Graph Convolutional Nets ?Similar to how message passing networks work, convolutional graph nets also claim to capture node representations while learning some information from the node's neighborhood.  However instead of performing aggregation over all nodes in the neighborhood, convolutional nets perform a first order approximation of  spectral graph convolution.
For a brief overview of convolution watch this great video 
﻿
﻿
Similar to Message Passing Graph Nets, our first hidden state is simply the vector of all node representations. 
H(0)=X\huge H^{(0)} = XH(0)=X﻿
There are many variants of Graph Convolutional Networks, however for the sake of simplicity we shall consider the simplest form as outlined in Simplifying Graph Convolutional Networks Wu et al. 2019. As outlined above Graph Convolutional Networks use approximations of spectral graph convolutions. A very simplistic view of convolutions is a weighted sum w.r.t to the filter. The simplest graph convolutional nets average a node representation with the feature vectors in its local neighborhood i.e., 
hv(t)=hvt−1di+1+∑j=1naij(di+1)(dj+1)hjt−1\huge h^{(t)}_v = \displaystyle \frac{h^{t-1}_{v}}{d_i + 1} + \sum_{j=1}^{n} \frac{a_{ij}}{\sqrt{(d_i + 1)(d_j + 1)}} h_{j}^{t-1}hv(t)​=di​+1hvt−1​​+j=1∑n​(di​+1)(dj​+1)​aij​​hjt−1​﻿
Even with just averaging, simple graphical convolutions has impressive results on most graphical tasks. The most general way of expressing convolutional graph nets is as follows:
hv=ϕ(xu,⊕v∈Nucuv ψ(xv))\huge h_v = \displaystyle \phi (x_u, \oplus_{v \in \mathcal{N}_u} c_{uv} \,\psi(x_v))hv​=ϕ(xu​,⊕v∈Nu​​cuv​ψ(xv​))﻿
where cuv\large c_{uv}cuv​﻿ represents the convolutional operation between nodes u\large uu﻿ and v\large vv﻿.
Let's have a look at some other variants of Graph Convolutional Networks:
Thomas Kipf and Max Welling use a first order approximation of spectral graph convolutions
One of the first works in extending convolutions to graphs Convolutional Neural Networks with Fast Localized Spectral Filtering by Defferrard et al. use the spectral approach by defining the graph convolution operation as the element wise Hadamard product between the Fourier basis of the filter and graph.
NOTE: We further discuss Graph Convolutional Networks in the following articles:
💡
A Brief Introduction to Residual Gated Graph Convolutional Networks
This article provides a brief overview of the Residual Gated Graph Convolutional Network architecture, complete with code examples in PyTorch Geometric and interactive visualizations using W&B. 
An Introduction to GraphSAGE
This article provides an overview of the GraphSAGE neural network architecture, complete with code examples in PyTorch Geometric, and visualizations using W&B. 
﻿
SummaryIn this article, we learned about Graph Convolutional Networks and walked through a simple definition and various other extensions in current literature. To see the full suite of W&B features, please check out this short 5 minutes guide.
If you want more reports covering graph neural networks with code implementations, let us know in the comments below or on our forum ✨!
Check out these other reports on Fully Connected covering other Graph Neural Networks-based topics and ideas.
Recommend Reading
Graph Neural Networks (GNNs) with Learnable Structural and Positional Representations
An in-depth breakdown of "Graph Neural Networks with Learnable Structural and Positional Representations" by Vijay Prakash Dwivedi, Anh Tuan Luu, Thomas Laurent, Yoshua Bengio and Xavier Bresson.
Using W&B with DeepChem: Molecular Graph Convolutional Networks
A quick tutorial on using W&B to track DeepChem molecular deep learning experiments
Part 1 – Introduction to Graph Neural Networks With GatedGCN
This article summarizes the need for Graph Neural Networks and analyzes one particular architecture – the Gated Graph Convolutional Network.
De Novo Molecule Generation with GCPNs using TorchDrug
How reinforcement learning, specifically graph convolutional policy networks, can help create brand new molecules to treat real world diseases
﻿
﻿
Add a comment