Assessment: Duplicate detection with Google GCP #1345

Is your feature request related to a problem?
External clients need to check whether a submission duplicates something already in a corpus. Today duplicate detection exists only inside the legacy RUN pre-filter (backend/app/services/assessment/prefilter/duplicate_detection.py): it is hardcoded to an OpenAI vector store and model (prefilter/constants.py), uses one fixed prompt, and has no endpoint a client can call.

Describe the solution you'd like
A dedicated assessment endpoint for duplicate detection, running on Google GCP (Vertex), with two verdict levels: strict and lenient.

Request contract:

{
  "query": "<submission text to check>",
  "config": {
    "strict":  {"config_id": "<uuid>", "version": 1},
    "lenient": {"config_id": "<uuid>", "version": 1}
  },
  "webhook_callback": "https://client.example.com/hook"
}

• Async delivery: the request is acknowledged with an id; both checks run in a Celery task and the combined result is POSTed to webhook_callback.

Response format, per level:

{
  "strict": {"is_duplicate": true, "citation": "<matching corpus passage>"},
  "lenient": [
    {"score": 4, "citation": "<matching corpus passage>"},
    {"score": 2, "citation": "<matching corpus passage>"}
  ]
}

• strict: one yes/no verdict, with the corpus passage that makes it a duplicate.
• lenient: a list of similar corpus entries, each with a similarity score and its citation.
• Each config enforces its shape through json_output_schema, so the model output is parsed, not free text.
• Kaapi owns the prompts and configs: we write the strict and lenient prompts, and create both configs ourselves through the existing POST /configs endpoint with the ASSESSMENT tag. Clients only reference them by config_id + version; they never author duplicate-detection prompts.
• Google GCP: the configs use provider google-gcp, grounded on a Vertex retrieval store holding the client corpus.
• Reuse: config resolution, Celery job execution, callback signing and delivery should reuse the existing assessment code paths rather than adding parallel ones.

Code layout
Keep it in its own module instead of growing the prefilter files:

• backend/app/api/routes/assessment/duplicate_detection.py: route
• backend/app/models/assessment/duplicate_detection.py: request, result and callback models
• backend/app/services/assessment/duplicate_detection/: service, prompts, response parsing
• Celery task alongside the existing assessment tasks in backend/app/celery/tasks/job_execution.py