feat(catalog): serve herus13 Apify actors on the platform key with capped billing - #657
Merged
Merged
Conversation
apify.web.scrape.job.start is priced free because the run bills later by the actor's own pricing, and nothing on the shared-key path meters that run. On treg's key it let any caller run any actor on treg's Apify account at no charge. It now needs the team's own Apify key; job.status and job.results stay open because per-team ownership already scopes them. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ctors Combines #641-#644 into one catalog change: nine run-sync entries over four herus13 actors, plus the lazada platform. The six Weibo entries serve on treg's key at a price observed on it. Lazada, Google Maps and TikTok Shop are own-key only: their actors bill run compute or a per-GB start fee that a per-result price cannot meter. Co-authored-by: Herus13 <bootforge.ai@gmail.com> Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Some upstreams take their spend bounds as query options rather than body fields (Apify's maxTotalChargeUsd, memory and timeout run options). A queryParams pin must be sent exactly once and is compared as the pinned value's type; own credentials keep the upstream contract. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
run-sync-get-dataset-items answers a bare array, so every Apify per_result call settled at its estimate whatever it returned. Count the rows, add an optional per-row-independent call_fee for the actor's start or compute charge, and require maxItems (1-200) on the platform key so the hold is the worst case. Apify's usageTotalUsd lags a finished run by minutes, so the response body is the settlement evidence. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Weibo, Google Maps, TikTok Shop and Lazada now settle on the platform key by counted dataset rows plus a flat call_fee (actor start, or Lazada's run compute). memory and timeout are pinned so the fee is fixed and a run ends before Apify's synchronous wait; TikTok Shop is held to keyword search. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
maxItems does not bind pay-per-event actors whose own input sets the row count, so a one-row hold could settle thousands of rows. Require the maxTotalChargeUsd run option Apify enforces (at most $1), hold it plus call_fee, accept only the run options each once in plain ASCII, and bill the hold when a run reaches its cap. Pin meta-ads enrichment off and bill the LinkedIn actor-start event on treg's key. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Row notes name maxTotalChargeUsd as the enforced spend limit; Lazada's compute fee is 0.015 under a 180-second timeout pin. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…orm runs Past Apify's 300-second synchronous wait a run answers 408 and keeps billing, so every Apify per_result platform call now names timeout <= 280. A cap under call_fee plus three rows would bill an empty answer in full, so it is refused. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
treg's upstream read timeout (call_timeout_s, 180 s) and the MCP client's 120 s end the call before a 280 s run finishes, releasing the hold unbilled while the run keeps billing. Bound timeout to 90 s and 30 s under call_timeout_s. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…s key Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A run that outlives its own timeout answers 400 run-failed with no rows, but Apify billed its events up to the caller's cap and the run id in that body reads the dataset. The caller chose the run's size and timeout, so the hold settles instead of releasing. The minimum-cap check compares micro-USD. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ricted Billing the whole cap on a TIMED-OUT 400 overcharged callers: a run that timed out after its start event cost Apify $0.00005 and would have billed the full cap. The loss treg absorbs stays bounded by the $1 cap, and keeping the Apify account's resource access Restricted stops anyone reading the unbilled run's rows by id. Examples now show the 90-second platform timeout. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…tors-2 # Conflicts: # frontend/e2e/landing.spec.ts # tests/test_asynctasks.py # tests/test_marketplace_call.py
…s the merge dropped Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This was referenced Sep 26, 2026
…tors-2 # Conflicts: # plugins/minimax/skills/treg/SKILL.md # plugins/treg/skills/treg/SKILL.md # skills/treg/SKILL.md
Each row's test_request ran on 2026-09-26 and Apify's chargedEventCounts matched its rate card. The TikTok ad library actor returns rows again, so it is verified with a captured example. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Adapters let treg.google.serp.maps and treg.weibo.post.detail choose the Apify actors, with the run's spend cap, timeout and memory fixed so the platform guard accepts the child call. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…of notes The actor's 2026-09-25 pricing no longer bills run compute to the caller (a 79-second run showed no platform usage), so its call_fee over-charged every call. Notes describe observations by price tier, not by the account that made them, and carry no run ids. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…tier Notes for plan-tiered actors record the events billed and that they matched the rate card for the key's tier, not the dollar figure that would name it; the repeated Weibo observation and the owner-meter notes go. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The LinkedIn jobs actor bills its actor-start event for every job title x location searched, so a flat call_fee under-billed any multi-query call. cost.call_fee_per names the body arrays whose lengths multiply the fee. The apify.yaml header no longer names a plan or calls the TikTok actor broken. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The actor bills an actor-start for every geo id it searches, and geoIds was undeclared, so it passed through unbilled. The row now takes its declared filters only (salary, easyApply, under10Applicants and industryIds added); places go in locations, which call_fee_per counts. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A run whose charges land exactly on maxTotalChargeUsd can be aborted by Apify and answer 400 with no rows, which releases the hold. Lazada's 10-product floor costs exactly its 0.05 minimum cap, so its example now uses 0.06 and the notes say to set the cap above the expected spend. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds nine Apify tools (six Weibo modes, Google Maps, TikTok Shop, Lazada) and serves them — and the three Apify rows already in the catalog — on the platform key with billing bounded by Apify's own spend cap. Generic
apify.web.scrape.job.startstays own-key only: its free price cannot meter an arbitrary actor.Supersedes #641, #642, #643 and #644. This PR carries the code the rows need as well, so there is nothing to merge first.
Why the billing needed code
Every Apify per-result row settled at its estimate:
run-sync-get-dataset-itemsanswers a bare JSON array, which no settlement rule counted. Live runs also showed three things a per-row price alone cannot handle:maxItemsdoes not bind pay-per-event actors whose own input sets the row count (maxItems=1with bodymax_items: 3returned and billed 3 rows). The run option Apify enforces on every billed event ismaxTotalChargeUsd.usageTotalUsdtrails a finished run by minutes (0.00005 at SUCCEEDED, 0.01505 45 s later), so it cannot be settlement evidence.What changed
Call path (
application/call/resolve.py,settle.py)platform_requestcan pinqueryParams.*singleton values; query values parse as plain ASCII only.per_resultcall on the platform key must sendmaxTotalChargeUsd(above 0, at most $1, at leastcall_fee+ three rows) andtimeout(at most 90 s, and 30 s undercall_timeout_s, so the run and its rows arrive before any wait gives up). OnlymaxTotalChargeUsd,maxItems,memoryandtimeoutare accepted, each once; dataset-view options (limit,format,unwind) are refused because they would make returned rows disagree with billed events.cost.call_fee. Settlement = returned rows × row price +call_fee; within two rows of the hold, the hold (the caller's cap was reached).Catalog (
apify.yaml)cost.call_fee(validated: Apify, per-result, USD) carries the per-run charge: Weibo start $0.00005 and Maps start $0.01 at pinned 1 GB, TikTok Shop two start events at its forced 2 GB ($0.004), LinkedIn jobs actor-start ($0.001 per job title × location searched:cost.call_fee_pernames the body arrays that multiply the fee). Lazada has none: since the actor's 2026-09-25 pricing it bills no start event and no run compute to the caller.memoryandtimeoutfor the new rows; TikTok Shopscrape_product_details: falseplusbody_allowlist(keeps the pricier product-detail event out); meta-adsenrichWithEcommerceData: false(an extra event per ad).body_allowlist, on every credential tier): the actor bills an actor-start per job title × location, whichcall_fee_percounts, and pergeoIdsentry, sogeoIdsis refused and places go inlocations. Filters that do not multiply starts (company, industry, salary, workplace/employment/experience, Easy Apply) stay declared.maxTotalChargeUsdas the spend limit. Lazada's actor, since 2026-09-25, refuses caps below $0.05 and returns at least 10 products.Routing (
adapters.yaml)treg.google.serp.mapsandtreg.weibo.post.detailnow include the Apify rows as candidates, the only two of these capabilities with a routing contract. The adapters fix the run's spend cap, timeout and memory so the child call passes the platform guard: Maps caps a routed call at 20 places (cap $0.25, no contact lookup), Weibo at one post (cap $0.05).Tests and CI
tests/test_asynctasks.py,tests/test_pinned_read_scope.py) now exercise Bright Data and CompanyEnrich, becauseapify.web.scrape.job.startno longer serves on the platform key; a new test proves it never reaches the relay.Docs:
docs/context/architecture/money.md("Apify dataset-row settlement"),catalog.md.Pricing on tiered actors
Google Maps, TikTok Shop and Meta ads price per Apify subscription tier. The catalog charges platform callers Apify's published default-plan (or, for Meta ads, BRONZE) rate, which is at or above what any paid plan pays, so the platform never bills below cost; own-key callers pay Apify directly at their own tier. This is deliberate list pricing, not an error.
Residual loss, bounded
A run that times out, fails or aborts answers 400 with no rows and releases; so does a caller disconnect or an answer over the 8 MiB evidence limit. Apify may still have billed events up to the cap, so the platform absorbs at most $1 per such call. Billing a timed-out run at the hold was tried and reverted: a run that timed out after its start event cost $0.00005 and would have billed the whole cap.
Operator precondition: the Apify account behind the platform key must keep general resource access Restricted. The 400 body names the run, and with public access anyone could read that unbilled run's dataset by id.
Evidence
Prices, 2026-09-26: every Apify per-result row's
test_requestran on the platform key and Apify'schargedEventCountsmatched its rate card; all twelve rows are nowsource: observed,confidence: verified.TikTok ads was marked upstream-broken (empty datasets); it returns rows again and is verified with a captured example. Routed calls through the adapters:
treg.google.serp.mapsreturned 20 places and billed $0.19;treg.weibo.post.detailreturned the post and billed $0.00505.Live on the Apify API:
pricingInfos) match every row's price, event and start fee.limit,format=csv, non-ASCII digits, repeated keys,timeoutover 90, meta-ads enrichment on, LinkedIngeoIds, TikTok Shopproduct_urls/detail mode.Reviewed in independent rounds with live verification; each blocking finding was fixed and re-reviewed.
Validation
openrouter.ai-judge.decideerrors are onmain).main(dashboard assets not built locally, hub, MCP OAuth, referrals, SEO, single-user); no failures specific to this branch.Follow-ups
TikTok ads: an undeclared
only_totalreturns one free count row that settlement bills as a result (caller overpays $0.003; no platform loss). Declare it or allowlist the row.Settle failed runs exactly from
chargedEventCountsthrough a deferred settle.A reconcile check that each custom-event actor still bills one event per returned row.
Accept an absent pinned field when the actor default equals the pin.
🤖 Generated with Claude Code