Epistemic Auditor for Large Language Models
AletheionGuard quantifies aleatoric (Q1) and epistemic (Q2) uncertainty in LLM outputs to detect hallucinations and assess response reliability.
# Minimal installation
pip install aletheion-guard
# With API server
pip install aletheion-guard[api]
# Full installation (all features)
pip install aletheion-guard[all]from aletheion_guard import EpistemicAuditor
# Initialize auditor (model weights included)
auditor = EpistemicAuditor()
# Audit any LLM response
prompt = "What is the capital of France?"
response = "The capital of France is Paris."
audit = auditor.audit(prompt, response)
print(f"Q1 (aleatoric): {audit.q1:.3f}") # Data ambiguity
print(f"Q2 (epistemic): {audit.q2:.3f}") # Model ignorance
print(f"Height: {audit.height:.3f}") # Proximity to truth
print(f"Verdict: {audit.verdict}") # ACCEPT | MAYBE | REFUSEDOutput:
Q1 (aleatoric): 0.023
Q2 (epistemic): 0.012
Height: 0.999
Verdict: ACCEPT
# Audit a response
aletheion-guard audit \
--prompt "What is 2+2?" \
--response "2+2 equals 4"
# Start API server
aletheion-guard serve --port 8000
# Show package info
aletheion-guard infoSeparates two types of uncertainty:
- Q1 (Aleatoric): Irreducible data noise/ambiguity
- Q2 (Epistemic): Model ignorance/hallucination risk
from aletheion_guard import epistemic_softmax
# Uncertainty-aware probability distributions
logits = model.get_logits("What is quantum computing?")
probs, uncertainty = epistemic_softmax(logits, return_uncertainty=True)
print(f"Q1: {uncertainty['q1']:.3f}, Q2: {uncertainty['q2']:.3f}")# pip install aletheion-guard[api]
from fastapi import FastAPI
from aletheion_guard.api import app
# Or use CLI
# aletheion-guard serve --host 0.0.0.0 --port 8000API Endpoints:
POST /v1/audit- Audit single responsePOST /v1/batch- Batch auditingPOST /v1/compare- Compare modelsGET /health- Health check
Model weights (~2.3MB) are bundled:
- Q1 Gate (aleatoric uncertainty)
- Q2 Gate (epistemic uncertainty)
- Height Gate (proximity to truth)
- Base Forces Network (4-force equilibrium)
audit = auditor.audit(prompt, llm_response)
if audit.verdict == "REFUSED":
return "I don't have enough confidence to answer this."
elif audit.q2 > 0.5:
return "This answer may be unreliable. Please verify."
else:
return llm_responseaudit = auditor.audit(query, rag_response)
if audit.q2 > 0.3:
# High epistemic uncertainty - retrieve more context
additional_docs = retriever.get_more_context(query)
improved_response = llm.generate(query, additional_docs)from aletheion_guard import EpistemicAuditor
auditor = EpistemicAuditor()
# Compare calibration across models
models = {
"gpt-4": gpt4_response,
"claude-3": claude_response,
"llama-3": llama_response
}
for model_name, response in models.items():
audit = auditor.audit(prompt, response)
print(f"{model_name}: Q2={audit.q2:.3f}, ECE={audit.ece:.3f}")AletheionGuard implements a pyramidal architecture for epistemic equilibrium:
┌─────────────────────────────────────┐
│ Epistemic Softmax Layer │ ← Uncertainty-aware predictions
├─────────────────────────────────────┤
│ Q1 Gate │ Q2 Gate │ Height │ ← Uncertainty quantification
├─────────────────────────────────────┤
│ Base Forces Network │ ← Memory, Pain, Choice, Exploration
├─────────────────────────────────────┤
│ Input Processor │ ← Text embeddings
└─────────────────────────────────────┘
Inspired by: aletheion-llm Based on: "How to Solve Skynet" research paper
# Core package (minimal dependencies)
pip install aletheion-guard
# With API server
pip install aletheion-guard[api]
# With monitoring (Prometheus, OpenTelemetry)
pip install aletheion-guard[monitoring]
# With ML utilities (PyTorch Lightning, Optuna)
pip install aletheion-guard[ml]
# With visualization (Matplotlib, Seaborn)
pip install aletheion-guard[viz]
# Development tools
pip install aletheion-guard[dev]
# All features
pip install aletheion-guard[all]auditor = EpistemicAuditor(
model_dir="/path/to/custom/weights"
)from aletheion_guard import EpistemicAuditor
auditor = EpistemicAuditor()
prompts = ["Question 1?", "Question 2?", "Question 3?"]
responses = ["Answer 1", "Answer 2", "Answer 3"]
for prompt, response in zip(prompts, responses):
audit = auditor.audit(prompt, response)
print(f"Q2: {audit.q2:.3f}, Verdict: {audit.verdict}")FROM python:3.11-slim
RUN pip install aletheion-guard[api]
EXPOSE 8000
CMD ["aletheion-guard", "serve", "--host", "0.0.0.0", "--port", "8000"]Each audit returns:
| Metric | Range | Description |
|---|---|---|
| Q1 | [0, 1] | Aleatoric uncertainty (data ambiguity) |
| Q2 | [0, 1] | Epistemic uncertainty (model ignorance) |
| Height | [0, 1] | Proximity to truth: h = 1 - √(Q1² + Q2²) |
| ECE | [0, 1] | Expected Calibration Error |
| Verdict | enum | ACCEPT | MAYBE | REFUSED |
# Clone repository
git clone https://github.com/AletheionAGI/AletheionGuard-Pypi.git
cd AletheionGuard-Pypi
# Install for development
pip install -e ".[dev]"
# Run tests
pytest tests/
# Format code
black src/
isort src/
# Type checking
mypy src/- Quick Start: https://aletheionguard.com/docs/quickstart
- API Reference: https://aletheionguard.com/docs/api/rest
- Architecture: https://aletheionguard.com/docs/concepts/epistemic
- Examples: https://aletheionguard.com/docs/examples/basic
- Research Paper: https://github.com/AletheionAGI/.github/blob/main/How_to_solve_Skynet__v_1_102.pdf
We welcome contributions! Please see our Contributing Guide.
AletheionGuard-Pypi is dual-licensed under:
- GNU AGPL-3.0 – for open source and non-commercial use.
- Aletheion Commercial License – for proprietary or commercial use.
Commercial use requires a paid license from AletheionAGI. For details, contact contact@aletheionagi.com.
- Website: aletheionguard.com
- Documentation: aletheionguard.com/docs
- GitHub: github.com/AletheionAGI/AletheionGuard-Pypi
- PyPI: pypi.org/project/aletheion-guard
- Discord: Join our community
If you use AletheionGuard in your research, please cite:
@software{aletheionguard2025,
title = {AletheionGuard: Epistemic Auditor for Large Language Models},
author = {Aletheion Research Collective},
year = {2025},
url = {https://github.com/AletheionAGI/AletheionGuard-Pypi},
version = {1.1.1}
}- ✅ Stable: Core API is stable and production-ready
- 🚀 Active Development: Regular updates and improvements
- 📦 PyPI: Official package available
- 🤖 API: Hosted API available at aletheionguard.com
Made with ❤️ by Aletheion Research Collective