Repository navigation
Establish what each candidate live model needs on agent-dev and costs #121
Description
Question
What does it take for the wrapper on agent-dev to run respondents on each candidate live model, and what does a pilot cost?
Facts to surface: whether EDSL 1.0.9's bedrock service can authenticate with AWS_BEARER_TOKEN_BEDROCK (boto3 version needed; boto3 is absent from ~/ep-venv), which model ids it accepts for us.anthropic.claude-haiku-4-5-20251001-v1:0, whether Model.available() and estimate_cost price Bedrock, OpenAI and Gemini models, what EDSL needs for openai and google services (env names), and the estimated cost of a 4-agent by 2-scenario pilot and of a 2-version, 4-task, 20-respondent conjoint on each candidate. Names of secrets only, never values. Record the EDSL and boto3 source revisions read.
Activity
beca-oc commentedon Nov 2, 2026
Resolved. Findings: docs/subconscious/research/live-model-needs-and-costs.md on research/live-model-needs (commit 284fb8e).
Bedrock (Haiku 4.5). EDSL 1.0.9's
aws_bedrock.pyimports boto3 at module load (line 3); boto3 is absent from~/ep-venv, soModel(..., service_name="bedrock")raisesInferenceServiceDependencyErrortoday. botocore 1.39.0 (2025-06-30) addedAWS_BEARER_TOKEN_BEDROCK; 1.39.12 fixed it, so pinboto3>=1.39.12. botocore prefers bearer over SigV4 when the token is set (handlers.py_should_prefer_bearer_auth). But EDSL's key lookup only recognisesAWS_SECRET_ACCESS_KEY/AWS_ACCESS_KEY_ID, so a non-emptyAWS_SECRET_ACCESS_KEYmust also be present or the first call fails with "No key found for service 'bedrock'". Theus.id passes through unchanged (_PROFILE_PREFIXES).OpenAI:
OPENAI_API_KEY, needsopenai>=2.44,<3(absent); default model gpt-4o. Gemini: EDSL readsGOOGLE_API_KEYonly, so exportGEMINI_API_KEYunder that name; needsgoogle-genai>=1.30,<2(absent).Cost:
estimate_costprices all three from Expected Parrot's price list (anonymous GET, 24 h disk cache; remote-inference flags do not affect it). Pilot (8 interviews): Haiku 4.5 us. $0.0098, gpt-4o-mini $0.0011, gemini-2.5-flash $0.0041. Conjoint (40 interviews, 160 prompts): $0.157 / $0.018 / $0.065. Nothing installed, no paid call made.beca-oc commentedon Nov 2, 2026
Closed as resolved; findings in docs/subconscious/research/live-model-needs-and-costs.md on branch research/live-model-needs.