You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem?
Another metrci we want to add is whether an assistant's answer actually follows its prompt — right language, right tone, answering what it should and refusing what it shouldn't. Need a metric for this.
Describe the solution you'd like
Research and prototype a judge that checks:
Language and tone match the prompt.
Answers what it should, refuses what it shouldn't.
Uses the fallback response when it doesn't know, instead of inventing facts.
Holds up against prompt injection.
Try it on a real org's assistant, check cost per question. Output: a scored Adherence to prompt metric that feeds the weighted average.
changed the title [-]Evaluation: Research adherence-to-prompt metric (instruction following)[/-][+]Evaluation: Metric for prompt adherence[/+]on Jul 13, 2026
Activity
[-]Evaluation: Research adherence-to-prompt metric (instruction following)[/-][+]Evaluation: Metric for prompt adherence[/+]