Skip to content

bench: rerun the ESP32-S3 suite on runtime 0.10.2 with per-suite pins - #43

Merged
asteinh merged 5 commits into
developfrom
bench/rerun-runtime-0.10.2
Sep 25, 2026
Merged

asteinh merged 5 commits into
developfrom
bench/rerun-runtime-0.10.2

Conversation

@asteinh

@asteinh asteinh commented Sep 25, 2026

Copy link
Copy Markdown
Member

Reruns the ESP32-S3 suite on runtime 0.10.2, which no longer sleeps between stages on ESP-IDF, and pins each suite to the core that produced its own results.

  • core-versions.json (format 2) pins compiler and runtime per suite. check_core_versions.py checks each suite's pins against that suite's summary and takes --suite for the checkout check, so a change that can only move one target's numbers reruns only that suite. The Cortex-M suite keeps its pins and results.
  • Every ESP32-S3 TiGrIS capture now ends with a BENCH_PROVENANCE: line naming the compiler and runtime it was built from. The collector derives the summary's producing revisions from those lines, so the summary stays a byte-identical reconstruction of its captures.

ESP32-S3, all result and accuracy gates pass:

Cell Before After
DS-CNN int8, ESP-NN 29.34 ms 20.36 ms
DS-CNN int8, TFLM 23.31 ms 23.31 ms
MobileNetV1 int8 256K 868.2 ms 844.9 ms
MobileNetV1 int8 128K 1188.4 ms 955.0 ms
MobileNetV1 int8 64K 1228.4 ms 902.4 ms
MobileNetV1 int8 32K 2608.4 ms 1611.1 ms
U-Net int8 12573.9 ms 12335.3 ms

A change that can only move one target's numbers now reruns only that suite. The ESP32-S3 summary records its producing compiler and runtime revisions so its pins can be checked.
The summary's producing revisions are derived from the captures, so the tracked summary stays a byte-identical reconstruction of them.
@asteinh
asteinh merged commit 3141199 into develop Sep 25, 2026
2 checks passed
@asteinh
asteinh deleted the bench/rerun-runtime-0.10.2 branch September 25, 2026 18:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant