[GSoC 2026] core(datamodel): populate the reconciled verdict for key-free domain/URL analyzers

[berardifra]

Description

Populate job.data_model with a reconciled verdict (evaluation + reliability) for the key-free
domain/URL analyzers that today produce a malicious/benign signal but throw it away because they carry no
DataModel hook. Once populated, the same verdict is read by the visualizer badge, the reconciliation
engine, pivots, and (in a follow-up chatbot PR) the assistant — one source of truth, no semantic fork.

This is the core-first part of the chatbot result-interpretation effort (PR B of the split; the
chatbot reader that consumes this lands separately). Refs #3892

What changes

• Shared classify() extracted from the DataModel visualizer into api_app/data_model_manager/classify.py
  (single source of the five presentation buckets + the 8/6 thresholds), with the bucket names in a
  DataModelVerdictBuckets(Choices) enum in data_model_manager/enums.py whose TRUSTED/MALICIOUS
  members derive from DataModelEvaluations; the visualizer now calls it — behaviour-preserving.
• 10 key-free DNS "malicious detector" analyzers (AdGuard, Quad9, CloudFlare, CleanBrowsing,
  UltraDNS, DNS4EU, Mullvad, Spamhaus_WQS, GoogleSafebrowsing, GoogleWebRisk) contribute
  evaluation=malicious only on a real hit, via a shared _do_create_data_model gate mixin + a
  declarative mapping_data_model migration (0195).
• PhishingArmy / Phishstats — same declarative pattern gated on a real listing (0196).
• Phishtank — evaluation=malicious, reliability 8 verified / 5 unverified (_update_data_model).
• Tranco — the one benign-direction source: popularity → evaluation=trusted, reliability banded by
  rank, with the top 1000 treated as a reliable allowlist (_update_data_model).

The load-bearing safety point: a $-prefixed mapping_data_model key writes its constant
unconditionally whenever a data model is created (analyzers_manager/models.py:92-95), so
{"$malicious":"evaluation"} would stamp MALICIOUS on every clean lookup. Each analyzer's
_do_create_data_model gate is what suppresses non-hits (miss / timeout / failure → no data model,
never trusted). This is unit-tested per analyzer, incl. the critical clean-lookup case.

Reliability table (single reviewable source; reliability = trust in the source):

source (on a hit)	evaluation	reliability	bucket
GoogleSafebrowsing, GoogleWebRisk	malicious	8	malicious
Spamhaus_WQS	malicious	7	malicious
AdGuard, Quad9, CloudFlare, CleanBrowsing, UltraDNS, DNS4EU, Mullvad	malicious	6	malicious
Phishtank (verified)	malicious	8	malicious
Phishtank (unverified)	malicious	5	suspicious
PhishingArmy, Phishstats	malicious	6	malicious
Tranco rank ≤ 1k	trusted	9	trusted
Tranco rank ≤ 10k / ≤ 100k / ranked	trusted	4 / 3 / 2	clean
any miss / timeout / note	— (no DataModel)	—	—

Two regimes, following review feedback:

• Top 1000 (reliability 9 → trusted) — treated as a reliable allowlist, and intended to outrank a
  malicious hit under the engine's average-reliability reconciliation. Rationale (@mlodic): in daily
  incident response false positives are the expensive failure mode, because the analyst time they burn
  costs more than the rare true positive they hide, so reliable allowlists must be enforced where
  available. This puts Tranco's top band at the same reliability as the pre-existing HuntingAbuseAPI
  allowlist (9).
• Below the top 1000 (reliability ≤4 → clean) — popularity is weak positive evidence: every malicious
  reliability (≥5) stays above it, so a popular-but-flagged domain still resolves to malicious.

The malicious tiers (8/7/6) reflect source authority and can be flattened on request.

Notes for reviewers (pre-empting two accurate observations from self-review):

• "Tranco is the only trusted source" is scoped to the verdicts this PR adds — HuntingAbuseAPI already
  maps $trusted/reliability 9 (allowlist, 0158) and is untouched here; it sits above the malicious
  range by design and does not affect the two regimes above.
• Pre-existing engine mechanics (informational, not a regression): metadata-only DataModels with a None
  evaluation default to reliability 5; a lone Tranco trusted from the lower bands (2–4) can be
  out-averaged by such a group and reconcile to no evaluation rather than clean. This is orthogonal
  to the malicious guarantee (malicious ≥6 always wins) and is existing EvaluationEngineModule
  behaviour, surfaced here for transparency.

Type of change

• New feature (non-breaking change which adds functionality).
• Chore (behaviour-preserving refactor: shared classify() extraction).

Checklist

• I have read and understood the rules about how to Contribute to this project
• The pull request is for the branch develop
• A new plugin (analyzer, connector, …) was added or changed, in which case:
  • I created the corresponding DataModel population for the changed analyzers following the
    documentation (declarative mapping_data_model + _do_create_data_model/_update_data_model hooks,
    mirroring the URLhaus / HuntingAbuseAPI precedents). No new analyzer was added — existing key-free
    analyzers gain DataModel hooks; the plugin sub-items about new-analyzer samples/FREE_TO_USE/url
    attributes do not apply.
  • Unit tests were added for every changed analyzer with all external calls mocked (no network in
    tests).
• I have inserted the copyright banner at the start of every new file.
• No new libraries were added.
• Linters (Ruff) gave 0 errors (check + format --check clean on all 25 files).
• I have added tests for the change (6 new test modules); all tests (new and old) gave 0 errors locally
  against a rebuilt test image (makemigrations --check clean; affected suite green, incl.
  engines_manager + visualizers_manager regression checks).
• GUI modified — N/A (the visualizer refactor is backend, behaviour-preserving; no rendered output
  changes).
• I will address any DeepSource / Django Doctors alerts raised on the PR.
• I will address raised Copilot issues.
• I have used LLMs (Claude Code) to help implement this PR, and I have reviewed and verified the
  generated code (design, per-analyzer implementation, and tests were reviewed; behaviour verified via the
  local test suite).

[mlodic]

Could you share a screenshot of a working execution of a DNS playbook analysis to show how data models are populated in the raw results of at least one of the changed analyzers?

[berardifra]

Could you share a screenshot of a working execution of a DNS playbook analysis to show how data models are populated in the raw results of at least one of the changed analyzers?

Here's a real run of the Dns playbook on malware.testcategory.com (job #35, 12/12 analyzers SUCCESS), with the CloudFlare_Malicious_Detector report expanded in the raw results:

The analyzer's report says "malicious": true, and the data_model it produced carries "evaluation": "malicious" with "reliability": 6, the value mapped for this analyzer in migration 0195.

The same screenshot also shows the gate working: Quad9_Malicious_Detector, UltraDNS_Malicious_Detector and DNS4EU_Malicious_Detector all answered "malicious": false on the same domain and produced no data model at all, so a clean lookup is never stamped malicious. The job's reconciled verdict is malicious / 6.

[mlodic]

thanks for the efforts and the explanations, wonderful result