Human-AI Review Deception

This repository contains the code and data used in the paper "Comparing Human and AI Deception in Online Reviews" by Linus Netze, Maximilian Maurer, Felix Soldner and Claudia Wagner. The paper is accepted for publication at KONVENS 2026.

We make the full dataset of human-written and LLM-generated reviews used in the paper available via the data folder in this repository, as well as in a structured way via the HuggingFace Hub: https://huggingface.co/datasets/gesis/human-ai-review-deception. The classification models used in the paper are not provided due to their large size (~250GB) and are only available upon request.

Structure

This repository is structured into multiple folders for the different steps as well as the data and code package used in the paper. Additional documentation is provided in the README of the respective directories.

• preprocessing: Contains notebooks and scripts used for preprocessing the dataset of human review datasets
• generator_selection: Contains the notebooks and scripts used for selecting the LLM which is used for review generation in the remainder of this paper.
• review_generation: Contains the notebooks and scripts used for generating reviews, postprocessing them and doing a small data analysis
• classification: Contains the notebooks and scripts used for training and evaluating classification models.
• data: Contains the data used by and produced in this paper, i.e. review datasets (human-written and LLM-generated), classification results, SHAP values
• src: Contains a python package providing the code dependencies used by the various scripts and notebooks (see here)

Getting data and models

Most of the data used and created in this paper is provided in this repository. The classification models on the other hand are not provided due to storage limiations as they have a size ~250GB. They are thus only available upon request.

Setting up the python environment

You need to setup a conda environment using the provided environment file:

conda env create -f environment.yml

Further this repo contains a python package containing various code dependencies required for executing the scripts/notebooks. This package can be installed with:

pip install .

Because I use pgf as export format for my matplotlib plots, the execution of notebooks creating such plots requires you to have LaTex installed on your system.

Citation

If you use this code or the data provided in this repository, please cite the following paper:

@inproceedings{netze2026human,
  title={Comparing Human and AI Deception in Online Reviews},
  author={Netze, Linus and Maurer, Maximilian and Soldner, Felix and Wagner, Claudia},
  booktitle={To Appear: Proceedings of the 22nd Conference on Natural Language Processing (KONVENS 2026)},
  year={2026}
}