Official implementation of “The Confidence Trap: Calibration Attacks for Graph Neural Networks”, accepted to KDD 2026.
This repository provides a unified benchmark for evaluating the robustness of Graph Neural Network (GNN) calibration methods under adversarial structural perturbations. It includes implementations of classical calibration methods, graph-specific calibration methods, and adversarial calibration attacks designed to manipulate confidence scores while preserving predicted labels.
Confidence calibration is critical for trustworthy machine learning systems, especially in safety-critical graph applications such as fraud detection, medical diagnosis, recommendation systems, and autonomous decision-making. A well-calibrated GNN should produce confidence scores that accurately reflect the likelihood of correctness.
However, calibrated GNNs can be vulnerable to calibration attacks, where an adversary perturbs the graph structure to distort model confidence while keeping the predicted label unchanged. This creates a subtle failure mode: the model may retain high classification accuracy while its confidence estimates become unreliable.
This repository studies the following question:
How robust are GNN calibration methods against adversarial graph perturbations?
To answer this question, we propose Unified Graph Calibration Attack (UGCA), a calibration attack framework that systematically degrades calibration quality under a label-preserving constraint.
- Unified benchmark for GNN calibration robustness
- Support for classical and graph-aware calibration methods
- Implementation of adversarial calibration attacks on graph structure
- Label-preserving attack setting for confidence manipulation
- Evaluation with Expected Calibration Error (ECE), accuracy, confidence reduction, and attack success rate
- Experiments on multiple graph datasets and calibration methods
The Confidence Trap: Calibration Attacks for Graph Neural Networks Cuong Dang, Jiahao Zhang, Hieu Ta Quang, Lu Cheng, Dung Le, Suhang Wang KDD 2026
@inproceedings{dang2026confidence,
title = {The Confidence Trap: Calibration Attacks for Graph Neural Networks},
author = {Dang, Cuong and Zhang, Jiahao and Ta Quang, Hieu and Cheng, Lu and Le, Dung and Wang, Suhang},
booktitle = {Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining},
year = {2026}
}This repository supports both classical post-hoc calibration methods and graph-specific calibration methods.
- Temperature Scaling (TS)
- Vector Scaling (VS)
- Matrix Scaling (MS)
- Ensemble Temperature Scaling (ETS)
Calibrated Graph Convolutional Networks A graph-aware calibration method that propagates confidence information over graph structure.
Graph Attention Temperature Scaling A calibration method that uses graph attention mechanisms to adaptively calibrate node-level predictions.
Data-Centric Graph Calibration A data-centric calibration method that adjusts graph structure or edge importance to improve calibration stability.
Graph Enhanced Temperature Scaling A graph-enhanced extension of temperature scaling that incorporates structural information into calibration.
Weighted Adaptive Temperature Scaling An adaptive temperature scaling method that assigns node-wise or sample-wise calibration weights.
Simple Calibration for GNNs A lightweight calibration framework for improving GNN confidence estimation with minimal overhead.
The main contribution of this repository is Unified Graph Calibration Attack (UGCA).
UGCA is designed to attack GNN confidence calibration while preserving predicted labels. Unlike standard adversarial attacks that aim to change model predictions, UGCA targets the model’s confidence estimates. This makes the attack harder to detect because classification accuracy can remain stable while calibration quality deteriorates.
UGCA contains four main components:
-
KL-Divergence Underconfidence Loss Encourages the predictive distribution to move toward a uniform distribution, producing stronger underconfidence attacks than margin-based objectives.
-
Reranking Mechanism Reranks candidate edge perturbations to avoid perturbations that are likely to flip the predicted label.
-
Hybrid Loss Switches between attack optimization and label-recovery optimization when the label-preservation constraint is violated.
-
Beam Search Expands the adversarial search space beyond greedy edge selection, reducing the risk of poor local optima.
Implementation of the proposed Unified Graph Calibration Attack (UGCA).
python ugca_calib_attack.py --calibration-method GETS --budget 5 --attack-nodes 100Influence-based graph calibration attack baseline.
python iga_calib_attack.pyRandom perturbation baseline for evaluating whether targeted attacks outperform naive structural perturbations.
python random_calib_attack.pyThe benchmark supports commonly used node classification datasets, including:
- Cora
- CoraML
- CiteSeer
- PubMed
- Photo
- Physics
- OGBN-Arxiv
Each dataset is split into:
- Training set
- Validation set
- Calibration set
- Test set
The GNN is trained on the training set, tuned on the validation set, calibrated on the calibration set, and evaluated on the test set.
UGCA follows a white-box attack setting, where the attacker has access to:
- Model gradients
- Model confidence scores
- Graph structure
- Target node predictions
The attacker is constrained to modify only edges connected to the target node. This setting models realistic graph attacks, such as a fraudulent account manipulating its direct connections in a financial transaction network.
The attack objective is to degrade calibration while preserving the predicted label:
Prediction before attack = Prediction after attack
Confidence before attack ≠ Confidence after attack
The benchmark evaluates calibration robustness using the following metrics.
Expected Calibration Error (ECE) measures the discrepancy between model confidence and empirical accuracy. Higher ECE indicates worse calibration.
Accuracy measures whether the model’s predicted labels remain correct. In calibration attacks, accuracy should ideally remain unchanged so that the attack specifically targets confidence reliability.
Confidence reduction measures the decrease in predicted confidence after an underconfidence attack.
Attack Success Rate (ASR) measures the percentage of attacked nodes for which the attack successfully changes confidence while preserving the predicted label.
Clone the repository:
git clone https://github.com/your-username/Graph-Neural-Network-Calibration-Attack.git
cd Graph-Neural-Network-Calibration-AttackInstall dependencies:
pip install -r requirements.txtpython benchmark_calibration_methods.pypython ugca_calib_attack.py \
--calibration-method GETS \
--budget 5 \
--attack-nodes 100python iga_calib_attack.py
python random_calib_attack.pyRun UGCA with Temperature Scaling:
python ugca_calib_attack.py --calibration-method TS --budget 5 --attack-nodes 100Run UGCA with CaGCN:
python ugca_calib_attack.py --calibration-method CaGCN --budget 5 --attack-nodes 100Run UGCA with DCGC:
python ugca_calib_attack.py --calibration-method DCGC --budget 5 --attack-nodes 100Graph-Neural-Network-Calibration-Attack/
├── benchmark_calibration_methods.py
├── ugca_calib_attack.py
├── iga_calib_attack.py
├── random_calib_attack.py
├── requirements.txt
├── README.md
├── data/
│ ├── cora/
│ ├── coraml/
│ ├── citeseer/
│ └── pubmed/
├── models/
│ ├── gcn.py
│ └── calibration_models.py
├── calibration/
│ ├── temperature_scaling.py
│ ├── vector_scaling.py
│ ├── matrix_scaling.py
│ ├── ensemble_temperature_scaling.py
│ ├── cagcn.py
│ ├── gats.py
│ ├── dcgc.py
│ ├── gets.py
│ ├── wats.py
│ └── simcalib.py
├── attacks/
│ ├── ugca.py
│ ├── iga.py
│ └── random_attack.py
├── utils/
│ ├── data_utils.py
│ ├── metrics.py
│ └── train_utils.py
└── results/
├── calibration_results/
└── attack_results/
The experiments show that:
- Calibration attacks can substantially increase ECE while preserving classification accuracy.
- Naively adapting image-domain calibration attacks to graphs is insufficient because graph perturbations are discrete and can easily flip predicted labels.
- UGCA achieves stronger confidence reduction than random and gradient-based baselines.
- KL-divergence loss, reranking, hybrid loss, and beam search each contribute to stronger attacks.
- Graph-aware and data-centric calibration methods generally provide stronger robustness under adversarial perturbations than classical calibration methods.
If you find this repository useful, please cite our paper:
@inproceedings{dang2026confidence,
title = {The Confidence Trap: Calibration Attacks for Graph Neural Networks},
author = {Dang, Cuong and Zhang, Jiahao and Ta Quang, Hieu and Cheng, Lu and Le, Dung and Wang, Suhang},
booktitle = {Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining},
year = {2026}
}This project is released under the MIT License.
This repository builds on prior work in graph neural network calibration, adversarial graph learning, and uncertainty estimation. We thank the authors of the calibration and graph attack methods used as baselines in this benchmark.