Self-supervised Learning on Graphs: Deep Insights and New Direction
read the original abstract
The success of deep learning notoriously requires larger amounts of costly annotated data. This has led to the development of self-supervised learning (SSL) that aims to alleviate this limitation by creating domain specific pretext tasks on unlabeled data. Simultaneously, there are increasing interests in generalizing deep learning to the graph domain in the form of graph neural networks (GNNs). GNNs can naturally utilize unlabeled nodes through the simple neighborhood aggregation that is unable to thoroughly make use of unlabeled nodes. Thus, we seek to harness SSL for GNNs to fully exploit the unlabeled data. Different from data instances in the image and text domains, nodes in graphs present unique structure information and they are inherently linked indicating not independent and identically distributed (or i.i.d.). Such complexity is a double-edged sword for SSL on graphs. On the one hand, it determines that it is challenging to adopt solutions from the image and text domains to graphs and dedicated efforts are desired. On the other hand, it provides rich information that enables us to build SSL from a variety of perspectives. Thus, in this paper, we first deepen our understandings on when, why, and which strategies of SSL work with GNNs by empirically studying numerous basic SSL pretext tasks on graphs. Inspired by deep insights from the empirical studies, we propose a new direction SelfTask to build advanced pretext tasks that are able to achieve state-of-the-art performance on various real-world datasets. The specific experimental settings to reproduce our results can be found in \url{https://github.com/ChandlerBang/SelfTask-GNN}.
This paper has not been read by Pith yet.
Forward citations
Cited by 4 Pith papers
-
SAOT: Self-Supervised Continual Graph Learning with Structure-Aware Optimal Transport
SAOT applies structure-aware optimal transport to capture global inter-node correspondences and uses cross-task distillation to retain prior structural knowledge, yielding accuracy gains of up to 15% on Products-CL in...
-
SSL4RL: Revisiting Self-supervised Learning as Intrinsic Reward for Visual-Language Reasoning
SSL4RL reformulates self-supervised learning objectives into dense, verifiable reward signals for RL-based fine-tuning of vision-language models, yielding performance gains on reasoning benchmarks.
-
OpenGLT: A Comprehensive Benchmark of Graph Neural Networks for Graph-Level Tasks
OpenGLT benchmark finds no single GNN architecture dominates graph-level tasks, with subgraph-based models strongest in expressiveness, graph learning and SSL models in robustness, node and pooling models in efficienc...
-
Automatic Self-supervised Learning for Social Recommendations
AusRec applies meta-learning to automatically weight multiple self-supervised tasks for improved social recommendation performance.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.