Discovery v1: add seed-based discovery, normalization, and review tooling

[fowler128]

Motivation

• Provide a reproducible input-acquisition layer to collect public creator/video/post candidates across YouTube, TikTok, and LinkedIn and normalize them for downstream market intelligence workflows.
• Support both Apify actor-based discovery and public fallback sources with a --dry-run mode for local testing and bootstrapping.
• Standardize output schema and basic scoring so future batches can be compared and triaged consistently.

Description

• Added discovery/apify_runner.py to load discovery/seed_queries.yaml, consult configs/platform_rules.yaml, call Apify actors (when configured) or fall back to YouTube RSS and DuckDuckGo site search, and write unified raw JSON to data/raw_candidates.json with a --dry-run option.
• Added discovery/normalize_to_market_intelligence.py to map raw JSON into output/market_intelligence.csv, preserving existing headers, appending needs_review and confidence_score, and deriving fields like niche and opening_hook.
• Added discovery/review_candidates.py to scan the normalized CSV and summarize/flag records missing critical metadata (creator_name, platform, source_url), transcripts, opening hooks, or CTAs.
• Added configuration, seed, example env, README, minimal requirements.txt, .gitignore, and sample outputs under data/ and output/ to demonstrate expected artifacts and run examples.

Testing

• Performed a dry-run smoke test by running python3 discovery/apify_runner.py --dry-run, which completed successfully and wrote data/raw_candidates.json (exit 0).
• Ran normalization with python3 discovery/normalize_to_market_intelligence.py --raw data/raw_candidates.json --output output/market_intelligence.csv, which produced output/market_intelligence.csv and exited successfully (exit 0).
• Executed the review script python3 discovery/review_candidates.py --csv output/market_intelligence.csv --sample-limit 5 which printed a summary and sample flagged rows and exited successfully (exit 0).

---

Codex Task

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
claude-code-practice	Ready	Preview, Comment	Apr 7, 2026 8:52am