Skip to content

android_screenshot: tool expects inline base64 but the app returns a media token ("No image data returned") #593

Description

@AYin-Z

Summary

android_screenshot (host-side tool) never returns an image with the current Android app. The tool parses the legacy inline-base64 shape, while the app's /screenshot bridge command already returns a relay media token — so the tool bails out with {"error": "No image data returned"}.

Versions

  • Plugin: 1.11.2 (also checked against server-v1.11.3 — the android_screenshot() body is identical there)
  • Android app: 1.16.1 sideload release (android-v1.16.1)

Evidence

Host side — plugin/tools/android_tool.py (~line 693):

data = _get("/screenshot")
if "error" in data:
    return json.dumps(data)
result = data.get("data", data)
img_b64 = result.get("image", "")
if not img_b64:
    return json.dumps({"error": "No image data returned"})

App side — BridgeCommandHandler.kt (~line 106) documents the contract:

 *  - `/screenshot` → returns `{media: "MEDIA:hermes-relay://<token>"}`

BridgeCommandHandler.kt (~line 1641) builds exactly that object, and ScreenCapture.kt (~line 594) produces the marker:

buildJsonObject { put("media", result.getOrNull()) }
// and
Result.success("MEDIA:hermes-relay://$token")

Observed response for GET /screenshot (bearer auth, loopback):

{"media": "MEDIA:hermes-relay://<token>"}

GET /media/<token> with the same bearer returns the actual PNG (1080x2400), so capture and media delivery both work — only the host tool's parsing is out of date.

Impact

android_screenshot is unusable against the current app. Accessibility-tree reads (android_read_screen / android_find_nodes) are unaffected.

Expected

Pick one canonical shape and make both sides agree — at the same commit, the app emits the media token while the tool expects base64. Either the tool should accept the media-token form (write the temp file from GET /media/<token>) or the relay should translate the app response into the legacy shape.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area:pluginplugin/ Python relay + toolsbugSomething isn't workingneeds-maintainer-reviewAutomated triage exhausted; needs a human

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions