This repo re-implements some popular Graph Neural Networks (GNNs) for learning purpose. The code depends only on PyTorch, but the datasets are taken from PyTorch Geometric.
Implemented models
- Graph Convolutional Network (GCN, ICLR 2017)
- Graph Attention Network (GAT, ICLR 2018)
- Simple Graph Convolution (SGC, ICML 2019)
conda create -n gnn python=3.10
conda activate gnn
conda install pytorch -c pytorch
# if you need CUDA
# conda install pytorch cudatoolkit -c pytorch
# optional
conda install pyg -c pyg
Note that you don't need torch-scatter
and torch-sparse
to use the datasets from PyTorch Geometric, except for NELL (to be confirmed).
Unless otherwise stated, hyperparameters are taken directly from the respective papers. Each experiment is ran 100 times, and the results are averaged.
python main.py --model gcn --dataset Cora # other options: CiteSeer, PubMed
Add --device cuda
if you want to train on GPU.
# For Cora, CiteSeer, and PubMed
python main.py --model gcn --dataset Cora --dropout 0.5 --l2_regularization 5e-4 --lr 1e-2 --num_epochs 200
# For NELL
python main.py --model gcn --dataset NELL --dropout 0.1 --l2_regularization 1e-5 --lr 1e-2 --hidden_dim 64 --num_epochs 200
Dataset | Cora | CiteSeer | PubMed | NELL |
---|---|---|---|---|
Paper | 81.5 | 70.3 | 79.0 | 66.0 |
This repo | 81.36 (±0.70) | 71.07 (±0.80) | 79.22 (±0.32) | 56.94 (±1.24) |
Related issues to GCN on NELL (possibly due to data preprocessing):
# For Cora and CiteSeer
python main.py --model gat --dataset Cora --dropout 0.6 --l2_regularization 5e-4 --lr 5e-3 --num_epochs 1000
# For PubMed (different from paper)
python main.py --model gat --dataset PubMed --dropout 0.6 --l2_regularization 5e-4 --lr 5e-3 --num_epochs 1000 --output_heads 8
Dataset | Cora | CiteSeer | PubMed |
---|---|---|---|
Paper | 83.0 | 72.5 | 79.0 |
This repo | 82.65 (±0.41) | 71.04 (±0.41) | 79.11 (±0.28) |
Related issues to GAT on CiteSeer and PubMed:
For PubMed, using hyperparameters provided by the paper --l2_regularization 1e-3 --lr 1e-2
, I can only achieve 77.36 (±0.52) accuracy.
# For Cora
python main.py --model sgc --dataset Cora --lr 0.2 --num_epochs 100 --weight_decay 1.3e-5
# For CiteSeer
python main.py --model sgc --dataset CiteSeer --lr 0.2 --num_epochs 150 --weight_decay 2.35e-5
# For PubMed
python main.py --model sgc --dataset PubMed --lr 0.2 --num_epochs 100 --weight_decay 7.4e-5
Dataset | Cora | CiteSeer | PubMed |
---|---|---|---|
Paper | 81.0 (±0.0) | 71.9 (±0.1) | 78.9 (±0.0) |
This repo | 81.03 (±0.05) | 71.80 (±0.00) | 78.96 (±0.05) |
Note: the performance can be slightly improved by increasing dropout (e.g. --dropout 0.6
) and training longer (e.g. num_epochs 200
)
The implementations here are tested against PyTorch Geometric's implementations for correctness. You will need to install torch-scatter
and torch-sparse
together with PyTorch Geometric. See their doc for installation instructions.
As of this writing, there are no pre-built torch-scatter
and torch-sparse
binaries for macOS ARM64 (Apple Silicon). You will need to install them from source
pip install torch-scatter torch-sparse
pip install torch-geometric
Once you have the pre-requisites, install pytest
via conda or pip then run:
python -m pytest