docs(domain-skills): add toolify, humanbenchmark, google-trends, google-autocomplete - #557
Conversation
…le-autocomplete Field-tested during keyword research for a cognitive-training SaaS product: - toolify.ai: correct /search/<query> URL pattern, and a scope warning that it's AI-tools-only (irrelevant results for non-AI niches). - humanbenchmark.com: full test catalog lives on the homepage (no /tests route), stable extraction selector, current 8-test list. - google-trends: the q=term1,term2,... compare-URL trick for relative volume, rendering/timing notes, and Rising-query noise caveat for low-volume seed terms. - google-autocomplete: the suggestqueries.google.com/complete/search endpoint for fast, unthrottled long-tail keyword discovery. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Adds new domain-skill documentation notes under agent-workspace/domain-skills/ to capture practical URL patterns, scraping/extraction snippets, and workflow caveats for SEO/keyword research sources used with browser-harness.
Changes:
- Documented Toolify’s
/search/<query>URL pattern and scope limitations for non–AI-tool niches. - Added Human Benchmark homepage-catalog scraping notes and a current test list for competitive gap checks.
- Added Google Trends compare-URL technique and SPA rendering/timing guidance.
- Added Google Suggest (autocomplete) endpoint and a small Python helper for fast keyword expansion.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| agent-workspace/domain-skills/toolify/search.md | Notes Toolify search URL pattern, scope caveat, and a basic DOM extraction approach. |
| agent-workspace/domain-skills/humanbenchmark/scraping.md | Documents homepage-based test catalog scraping and enumerates current test set. |
| agent-workspace/domain-skills/google-trends/explore.md | Captures Trends compare-URL trick, SPA timing notes, and Rising-query interpretation caveats. |
| agent-workspace/domain-skills/google-autocomplete/suggest-api.md | Documents the Suggest endpoint plus a simple Python snippet and expansion workflow. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| Comma-separate terms in `q=`, no URL-encoding needed for the comma itself (spaces still need `%20` or `+`): | ||
|
|
||
| ``` | ||
| https://trends.google.com/trends/explore?date=today%2012-m&geo=US&q=term one,term two,term three,term four,term five&hl=en |
There was a problem hiding this comment.
2 issues found across 4 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="agent-workspace/domain-skills/toolify/search.md">
<violation number="1" location="agent-workspace/domain-skills/toolify/search.md:28">
P3: The documented extraction snippet only returns matching `<a>` nodes, not the card titles it claims to extract. Consumers that use this result as a list of titles will receive DOM elements instead of strings; mapping each anchor to its heading text (and optionally its href) would make the example usable.</violation>
</file>
<file name="agent-workspace/domain-skills/google-trends/explore.md">
<violation number="1" location="agent-workspace/domain-skills/google-trends/explore.md:20">
P2: The example URL contains literal spaces in the `q=` parameter (`term one,term two,...`), but the preceding line explicitly states that spaces need `%20` or `+` encoding. This contradiction will confuse readers and many HTTP clients will reject or re-encode the URL unpredictably. Encode the spaces to match your own guidance, e.g. `q=term%20one,term%20two,term%20three,term%20four,term%20five`.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Fix all with cubic | Re-trigger cubic
| Comma-separate terms in `q=`, no URL-encoding needed for the comma itself (spaces still need `%20` or `+`): | ||
|
|
||
| ``` | ||
| https://trends.google.com/trends/explore?date=today%2012-m&geo=US&q=term one,term two,term three,term four,term five&hl=en |
There was a problem hiding this comment.
P2: The example URL contains literal spaces in the q= parameter (term one,term two,...), but the preceding line explicitly states that spaces need %20 or + encoding. This contradiction will confuse readers and many HTTP clients will reject or re-encode the URL unpredictably. Encode the spaces to match your own guidance, e.g. q=term%20one,term%20two,term%20three,term%20four,term%20five.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At agent-workspace/domain-skills/google-trends/explore.md, line 20:
<comment>The example URL contains literal spaces in the `q=` parameter (`term one,term two,...`), but the preceding line explicitly states that spaces need `%20` or `+` encoding. This contradiction will confuse readers and many HTTP clients will reject or re-encode the URL unpredictably. Encode the spaces to match your own guidance, e.g. `q=term%20one,term%20two,term%20three,term%20four,term%20five`.</comment>
<file context>
@@ -0,0 +1,39 @@
+Comma-separate terms in `q=`, no URL-encoding needed for the comma itself (spaces still need `%20` or `+`):
+
+```
+https://trends.google.com/trends/explore?date=today%2012-m&geo=US&q=term one,term two,term three,term four,term five&hl=en
+```
+
</file context>
| https://trends.google.com/trends/explore?date=today%2012-m&geo=US&q=term one,term two,term three,term four,term five&hl=en | |
| https://trends.google.com/trends/explore?date=today%2012-m&geo=US&q=term%20one,term%20two,term%20three,term%20four,term%20five&hl=en |
| Card titles are inside `h2`/`h3`/`h4` wrapped by an `<a>`; a plain `querySelectorAll("a")` scan filtering for a heading child works, e.g.: | ||
|
|
||
| ```js | ||
| Array.from(document.querySelectorAll("a")).filter(a => a.querySelector("h2,h3,h4")) |
There was a problem hiding this comment.
P3: The documented extraction snippet only returns matching <a> nodes, not the card titles it claims to extract. Consumers that use this result as a list of titles will receive DOM elements instead of strings; mapping each anchor to its heading text (and optionally its href) would make the example usable.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At agent-workspace/domain-skills/toolify/search.md, line 28:
<comment>The documented extraction snippet only returns matching `<a>` nodes, not the card titles it claims to extract. Consumers that use this result as a list of titles will receive DOM elements instead of strings; mapping each anchor to its heading text (and optionally its href) would make the example usable.</comment>
<file context>
@@ -0,0 +1,29 @@
+Card titles are inside `h2`/`h3`/`h4` wrapped by an `<a>`; a plain `querySelectorAll("a")` scan filtering for a heading child works, e.g.:
+
+```js
+Array.from(document.querySelectorAll("a")).filter(a => a.querySelector("h2,h3,h4"))
+```
</file context>
Summary
toolify.ai: correct/search/<query>URL pattern (the?q=query-string form doesn't work), plus a scope warning — it's an AI-tools-only directory and returns irrelevant results for non-AI niches (tested on "focus training" / "attention training").humanbenchmark.com: the full test catalog lives on the homepage, not a/testsroute (that 404s); stable extraction snippet; current 8-test list documented for future competitive-gap checks.google-trends: theq=term1,term2,...compare-URL trick to get relative search-interest on one normalized 0-100 chart (much better thanpytrends, which 429s quickly), rendering/timing notes for the SPA, and a caveat that Rising-query panels get noisy for low-volume/ambiguous seed terms.google-autocomplete: thesuggestqueries.google.com/complete/search?client=firefoxendpoint for fast, unthrottled long-tail keyword discovery (no auth, no rate limit hit across dozens of calls).Test plan
http_getduring the research session and outputs were verified (screenshots + JSON responses).🤖 Generated with Claude Code
Summary by cubic
Adds four domain-skill docs for keyword research: Google Autocomplete, Google Trends, Human Benchmark, and Toolify search. These notes clarify correct endpoints, compare URL patterns, stable selectors, and scope limits to avoid dead ends.
google-autocomplete/suggest-api.md: documentssuggestqueries.google.com/complete/search?client=firefoxJSON endpoint and a quick expansion strategy; notes light limits and a polite delay.google-trends/explore.md: shows theq=term1,term2,...compare URL for a shared 0–100 chart; SPA load timing and Rising-panel caveats; recommends UI overpytrendsdue to 429s.humanbenchmark/scraping.md: confirms all tests are listed on the homepage (no/testsroute) and adds a stable selector; records the current 8-test list.toolify/search.md: correct/search/<query>path pattern (query-string form fails) and a scope warning that this directory isn’t useful outside its niche; adds a simple card selector.Written for commit 5410cfa. Summary will update on new commits.