Add repeatable content-intelligence workflow outputs and templates

.env

@@ -0,0 +1,7 @@
+# Apify API token (optional if only using public fallback sources)
+APIFY_TOKEN=
+
+# Optional actor IDs by platform. If unset, runner falls back to public discovery sources.
+APIFY_ACTOR_YOUTUBE=
+APIFY_ACTOR_TIKTOK=
+APIFY_ACTOR_LINKEDIN=

.env.example

@@ -0,0 +1,7 @@
+# Apify API token (optional if only using public fallback sources)
+APIFY_TOKEN=
+
+# Optional actor IDs by platform. If unset, runner falls back to public discovery sources.
+APIFY_ACTOR_YOUTUBE=
+APIFY_ACTOR_TIKTOK=
+APIFY_ACTOR_LINKEDIN=

README.md

@@ -0,0 +1,65 @@
+# Content Intelligence Repo
+
+## Discovery v1
+
+Discovery v1 adds an input-acquisition layer that collects public creator/video/post candidates across YouTube, TikTok, and LinkedIn-related search results, then normalizes them into `output/market_intelligence.csv`.
+
+### Files
+- `discovery/seed_queries.yaml`: lane-based seed queries and platform toggles.
+- `configs/platform_rules.yaml`: platform discovery modes and actor env bindings.
+- `discovery/apify_runner.py`: candidate collection runner with Apify + public fallback.
+- `discovery/normalize_to_market_intelligence.py`: maps raw JSON to flat CSV.
+- `discovery/review_candidates.py`: record quality checks and review summary.
+- `data/raw_candidates.json`: persisted raw candidate payload.
+
+### Environment
+Copy `.env.example` to `.env` and set values as needed:
+
+```bash
+cp .env.example .env
+```
+
+Required/optional variables:
+- `APIFY_TOKEN` (optional if using only fallback sources)
+- `APIFY_ACTOR_YOUTUBE` (optional)
+- `APIFY_ACTOR_TIKTOK` (optional)
+- `APIFY_ACTOR_LINKEDIN` (optional)
+
+### Install
+```bash
+python3 -m pip install -r requirements.txt
+```
+
+### Run discovery
+Dry-run mode (no external calls):
+```bash
+python3 discovery/apify_runner.py --dry-run
+```
+
+Live mode (uses Apify actors if configured; otherwise public fallback sources):
+```bash
+python3 discovery/apify_runner.py
+```
+
+### Normalize to market intelligence CSV
+```bash
+python3 discovery/normalize_to_market_intelligence.py
+```
+
+Behavior:
+- Preserves existing header order when `output/market_intelligence.csv` already exists.
+- Appends `needs_review` and `confidence_score` if missing.
+- Leaves unavailable fields blank.
+
+### Review candidate quality
+```bash
+python3 discovery/review_candidates.py
+```
+
+Checks include:
+- incomplete core metadata (`creator_name`, `platform`, `source_url`)
+- missing transcript/summary
+- missing opening hook
+- missing CTA
+
+The script prints a summary + sample flagged records for triage.

configs/platform_rules.yaml

@@ -0,0 +1,29 @@
+platforms:
+  youtube:
+    discovery_mode: "apify_or_rss"
+    actor_env: "APIFY_ACTOR_YOUTUBE"
+    defaults:
+      content_type: "video"
+      platform: "YouTube"
+  tiktok:
+    discovery_mode: "apify_or_search"
+    actor_env: "APIFY_ACTOR_TIKTOK"
+    defaults:
+      content_type: "short_video"
+      platform: "TikTok"
+  linkedin:
+    discovery_mode: "apify_or_search"
+    actor_env: "APIFY_ACTOR_LINKEDIN"
+    defaults:
+      content_type: "post_or_video"
+      platform: "LinkedIn"
+normalization:
+  csv_output: "output/market_intelligence.csv"
+  required_fields:
+    - creator_name
+    - platform
+    - source_url
+  review_fields:
+    - transcript
+    - opening_hook
+    - cta

data/raw_candidates.json

@@ -0,0 +1,126 @@
+{
+  "generated_at": 1775355334,
+  "dry_run": true,
+  "records": [
+    {
+      "platform": "youtube",
+      "lane": "legal_ops",
+      "query": "legal operations workflow",
+      "creator_name": "DryRun Creator",
+      "video_title_or_topic": "Dry run candidate for legal operations workflow",
+      "source_url": "https://example.com/dry-run",
+      "posting_date": "",
+      "discovery_source": "dry_run"
+    },
+    {
+      "platform": "youtube",
+      "lane": "legal_ops",
+      "query": "contract lifecycle bottlenecks",
+      "creator_name": "DryRun Creator",
+      "video_title_or_topic": "Dry run candidate for contract lifecycle bottlenecks",
+      "source_url": "https://example.com/dry-run",
+      "posting_date": "",
+      "discovery_source": "dry_run"
+    },
+    {
+      "platform": "youtube",
+      "lane": "legal_ops",
+      "query": "legal ops dashboard",
+      "creator_name": "DryRun Creator",
+      "video_title_or_topic": "Dry run candidate for legal ops dashboard",
+      "source_url": "https://example.com/dry-run",
+      "posting_date": "",
+      "discovery_source": "dry_run"
+    },
+    {
+      "platform": "youtube",
+      "lane": "grc",
+      "query": "GRC operating model",
+      "creator_name": "DryRun Creator",
+      "video_title_or_topic": "Dry run candidate for GRC operating model",
+      "source_url": "https://example.com/dry-run",
+      "posting_date": "",
+      "discovery_source": "dry_run"
+    },
+    {
+      "platform": "youtube",
+      "lane": "grc",
+      "query": "risk and compliance process improvement",
+      "creator_name": "DryRun Creator",
+      "video_title_or_topic": "Dry run candidate for risk and compliance process improvement",
+      "source_url": "https://example.com/dry-run",
+      "posting_date": "",
+      "discovery_source": "dry_run"
+    },
+    {
+      "platform": "youtube",
+      "lane": "grc",
+      "query": "governance risk compliance metrics",
+      "creator_name": "DryRun Creator",
+      "video_title_or_topic": "Dry run candidate for governance risk compliance metrics",
+      "source_url": "https://example.com/dry-run",
+      "posting_date": "",
+      "discovery_source": "dry_run"
+    },
+    {
+      "platform": "youtube",
+      "lane": "ai_governance",
+      "query": "AI governance framework",
+      "creator_name": "DryRun Creator",
+      "video_title_or_topic": "Dry run candidate for AI governance framework",
+      "source_url": "https://example.com/dry-run",
+      "posting_date": "",
+      "discovery_source": "dry_run"
+    },
+    {
+      "platform": "youtube",
+      "lane": "ai_governance",
+      "query": "enterprise AI risk controls",
+      "creator_name": "DryRun Creator",
+      "video_title_or_topic": "Dry run candidate for enterprise AI risk controls",
+      "source_url": "https://example.com/dry-run",
+      "posting_date": "",
+      "discovery_source": "dry_run"
+    },
+    {
+      "platform": "youtube",
+      "lane": "ai_governance",
+      "query": "AI policy implementation",
+      "creator_name": "DryRun Creator",
+      "video_title_or_topic": "Dry run candidate for AI policy implementation",
+      "source_url": "https://example.com/dry-run",
+      "posting_date": "",
+      "discovery_source": "dry_run"
+    },
+    {
+      "platform": "youtube",
+      "lane": "operations_leadership",
+      "query": "operations leadership systems",
+      "creator_name": "DryRun Creator",
+      "video_title_or_topic": "Dry run candidate for operations leadership systems",
+      "source_url": "https://example.com/dry-run",
+      "posting_date": "",
+      "discovery_source": "dry_run"
+    },
+    {
+      "platform": "youtube",
+      "lane": "operations_leadership",
+      "query": "workflow integrity",
+      "creator_name": "DryRun Creator",
+      "video_title_or_topic": "Dry run candidate for workflow integrity",
+      "source_url": "https://example.com/dry-run",
+      "posting_date": "",
+      "discovery_source": "dry_run"
+    },
+    {
+      "platform": "youtube",
+      "lane": "operations_leadership",
+      "query": "business systems strategy",
+      "creator_name": "DryRun Creator",
+      "video_title_or_topic": "Dry run candidate for business systems strategy",
+      "source_url": "https://example.com/dry-run",
+      "posting_date": "",
+      "discovery_source": "dry_run"
+    }
+  ]
+}