Skip to content

perf: guard the complete core benchmark suite #51

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tiye merged 1 commit into from
Aug 22, 2026
Merged

perf: guard the complete core benchmark suite #51

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

QuickCoffee 是一台以 Rust 编写、受 CoffeeScript 启发的字节码脚本引擎。它保留紧凑、可读的表达式语法，却不兼容 JavaScript：没有原型链、`this`、`eval` 或嵌入 JavaScript。

当前实现遵循 [RFCs/0000-project-scope.md](RFCs/0000-project-scope.md) 至 [RFCs/0108-qtest-showcase-corpus.md](RFCs/0108-qtest-showcase-corpus.md)。
当前实现遵循 [RFCs/0000-project-scope.md](RFCs/0000-project-scope.md) 至 [RFCs/0109-qbench-core-suite.md](RFCs/0109-qbench-core-suite.md)。

```coffee
square = (x) -> x * x
Expand Down
2 changes: 1 addition & 1 deletion RFCs/0000-project-scope.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,4 @@ QuickCoffee 是一个 Rust 实现的、受 CoffeeScript 2016 启发的脚本引

本仓库中的测试即 0.1 的语义基线。对语法或运行时的新增特性必须先以 RFC 补充定义，并至少添加：成功测试、错误测试及字节码验证测试。

当前已实现的后续语义与工具 RFC 延伸至 RFC 0108；其中 RFC 0077 定义 JSON 输出、RFC 0079 定义 TAP 输出、RFC 0080 定义 CLI 字节码指纹、RFC 0081 定义可机器读取的基准输出、RFC 0082 规范化指纹编码、RFC 0083 定义 Markdown 文学编程产物、RFC 0084 定义嵌入上下文 fuel 控制、RFC 0085 定义可执行 Rust 嵌入示例、RFC 0086 定义 crate 发布元数据、RFC 0094 定义 qdocco 最终值门禁、RFC 0095 定义字符串步进迭代、RFC 0096 定义其性能基准、RFC 0097 定义 `do` 参数转发、RFC 0098 定义 RFC 索引门禁、RFC 0099 定义 `!` 否定别名、RFC 0100 定义有符号 `by` 步长、RFC 0101 定义 qdocco 原子输出、RFC 0102 定义 qtest 规范文件去重、RFC 0103 定义 qbench schema 版本、RFC 0104 定义 qdocco 块注释代码保留、RFC 0105 定义 qbench 重复采样中位数、RFC 0106 定义 crate 发布包验收门禁、RFC 0107 定义 release qbench 持续门禁、RFC 0108 定义 qtest 可执行示例语料，均不改变脚本语言值模型的原型无关约束。
当前已实现的后续语义与工具 RFC 延伸至 RFC 0109；其中 RFC 0077 定义 JSON 输出、RFC 0079 定义 TAP 输出、RFC 0080 定义 CLI 字节码指纹、RFC 0081 定义可机器读取的基准输出、RFC 0082 规范化指纹编码、RFC 0083 定义 Markdown 文学编程产物、RFC 0084 定义嵌入上下文 fuel 控制、RFC 0085 定义可执行 Rust 嵌入示例、RFC 0086 定义 crate 发布元数据、RFC 0094 定义 qdocco 最终值门禁、RFC 0095 定义字符串步进迭代、RFC 0096 定义其性能基准、RFC 0097 定义 `do` 参数转发、RFC 0098 定义 RFC 索引门禁、RFC 0099 定义 `!` 否定别名、RFC 0100 定义有符号 `by` 步长、RFC 0101 定义 qdocco 原子输出、RFC 0102 定义 qtest 规范文件去重、RFC 0103 定义 qbench schema 版本、RFC 0104 定义 qdocco 块注释代码保留、RFC 0105 定义 qbench 重复采样中位数、RFC 0106 定义 crate 发布包验收门禁、RFC 0107 定义 release qbench 持续门禁、RFC 0108 定义 qtest 可执行示例语料、RFC 0109 定义 qbench 核心负载全套护栏，均不改变脚本语言值模型的原型无关约束。
18 changes: 18 additions & 0 deletions RFCs/0109-qbench-core-suite.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# RFC 0109：qbench 核心负载全套护栏

- 状态：已采纳
- 依赖：RFC 0045、RFC 0081、RFC 0105、RFC 0107

## 约束

`cargo bench --bench core` 中的每一个命名负载都必须同时出现在 `qbench --json` 的内建负载集合中。两者使用相同的源程序和最终值期望；core benchmark 可以使用更大的迭代次数，而 qbench 负责较小、可在 CI 中运行的编译、验证和执行语义护栏。

qbench 输出必须为每个负载产生恰好一条记录，记录包含 schema、版本、负载名、迭代数、重复数、期望值和三个阶段的纳秒计时。每轮计时前后仍需经过 RFC 0045 的最终值检查；任何负载编译、验证或执行失败都使命令失败。新增 core benchmark 负载时，必须同步加入 qbench 并更新 CLI 集成断言。

## 性能口径

`make check` 使用 `--iterations 1 --repeat 3` 覆盖全套负载，保证优化后的 VM 路径不会绕过 release 门禁；正式性能数据仍由 `cargo bench --bench core` 和 RFC 0096 的重复采样方法产生。该门禁不设机器相关的时间阈值，只验证语义和输出结构。

## 验收

CLI 集成测试必须断言全套负载名称各出现一次，并检查每条 JSON 记录的 schema、版本、重复数、期望值及 compile/verify/execute 字段。
140 changes: 140 additions & 0 deletions src/bin/qbench.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,146 @@ const WORKLOADS: &[Workload] = &[
source: "sum = 0\nfor n, index in [1...100] by -3 then sum += n + index\nsum",
expected: "3333",
},
Workload {
name: "postfix-loops",
source: "sum = 0\ni = 0\ni = i + 1 while i < 100\nsum + i",
expected: "100",
},
Workload {
name: "array-slices",
source: "items = [0...100]\nsum = 0\ni = 0\nwhile i < 100\n slice = items[10...90]\n sum = sum + slice[0] + slice[79]\n i = i + 1\nsum",
expected: "9900",
},
Workload {
name: "existence-tests",
source: "value = nil\nsum = 0\ni = 0\nwhile i < 100\n sum = sum + (if value? then 0 else 1)\n i = i + 1\nsum",
expected: "100",
},
Workload {
name: "existential-assignment",
source: "value = 1\nsum = 0\ni = 0\nwhile i < 100\n value ?= 2\n sum = sum + value\n i = i + 1\nsum",
expected: "100",
},
Workload {
name: "name-updates",
source: "i = 0\nsum = 0\nwhile i < 100\n sum += i\n i++\nsum",
expected: "4950",
},
Workload {
name: "floor-modulo",
source: "sum = 0\ni = -100\nwhile i < 100\n sum += i // 3\n sum += i %% 7\n i += 1\nsum",
expected: "500",
},
Workload {
name: "bitwise",
source: "sum = 0\ni = -100\nwhile i < 100\n sum += (i & 31) ^ (i << 1)\n i += 1\nsum",
expected: "-196",
},
Workload {
name: "multiline-strings",
source: "message = \"alpha\n beta\"\nlen(message)",
expected: "10",
},
Workload {
name: "string-iteration",
source: "sum = 0\nfor character, index in 'a☕中' then sum += index\nsum",
expected: "3",
},
Workload {
name: "string-escapes",
source: "message = \"A\\x42\\u{43}\"\nlen(message) + (if message == 'ABC' then 1 else 0)",
expected: "4",
},
Workload {
name: "string-indexing",
source: "text = 'a☕中'\nsum = 0\ni = 0\nwhile i < 100\n sum += len(text[1..2]) + (if text[1] == '☕' then 1 else 0)\n i += 1\nsum",
expected: "300",
},
Workload {
name: "multiline-collections",
source: "values = [\n 1\n 2\n 3\n]\nrecord = {\n first: 1\n second: 2\n}\nvalues[2] + record.first + record.second",
expected: "6",
},
Workload {
name: "indented-maps",
source: "record =\n first: 1\n nested:\n second: 2\nrecord.nested.second + record.first",
expected: "3",
},
Workload {
name: "implicit-calls",
source: "add = (left, right) -> left + right\nanswer = add 20, 22\nanswer",
expected: "42",
},
Workload {
name: "execution-stats",
source: "sum = 0\ni = 0\nwhile i < 100\n sum += i\n i++\nsum",
expected: "4950",
},
Workload {
name: "constant-folding",
source: "value = (1 + 2 * 3) == 7\nvalue",
expected: "true",
},
Workload {
name: "bare-lambda",
source: "base = 1\nadd = n -> n + base\nsum = 0\nfor n in [1...50] then sum = sum + add(n)\nsum",
expected: "1274",
},
Workload {
name: "stepped-iteration",
source: "sum = 0\nfor n in [1...100] by 3 then sum = sum + n\nsum",
expected: "1617",
},
Workload {
name: "for-collection",
source: "values = for n in [1...100] when n % 3 == 0 then n * 2\nlen(values)",
expected: "33",
},
Workload {
name: "postfix-comprehension",
source: "values = n * 2 for n in [1...100]\nsum = 0\nfor n in values then sum = sum + n\nsum",
expected: "9900",
},
Workload {
name: "for-pattern-bindings",
source: "pairs = for n in [1...100] then [n, n + 1]\nsum = 0\nfor [left, right] in pairs then sum = sum + left + right\nsum",
expected: "9999",
},
Workload {
name: "maps-and-control",
source: "record = {a: 1, b: 2, c: 3}\nsum = 0\nfor own key, value of record when value > 1 then sum = sum + value\ntry sum ? 0 catch error then 0",
expected: "5",
},
Workload {
name: "soak-access",
source: "record = {answer: 1}\nnone = nil\nsum = 0\ni = 0\nwhile i < 100\n sum = sum + record?.answer + (none?[i] ? 0)\n i = i + 1\nsum",
expected: "100",
},
Workload {
name: "nested-destructuring",
source: "sum = 0\ni = 0\nwhile i < 100\n [first, {point: [x, y]}] = [1, {point: [2, 3]}]\n sum = sum + first + x + y\n i = i + 1\nsum",
expected: "600",
},
Workload {
name: "destructuring-rest",
source: "sum = 0\ni = 0\nwhile i < 100\n [head, tail...] = [1, 2, 3, 4]\n sum += head + len(tail)\n i += 1\nsum",
expected: "400",
},
Workload {
name: "chained-comparisons",
source: "low = 0\nmiddle = 1\nhigh = 2\nsum = 0\ni = 0\nwhile i < 100\n sum = sum + (if low < middle < high then 1 else 0)\n i = i + 1\nsum",
expected: "100",
},
Workload {
name: "destructuring-parameters",
source: "scale = ([left, right], {factor}) -> (left + right) * factor\nsum = 0\ni = 0\nwhile i < 100\n sum = sum + scale([1, 2], {factor: 3})\n i = i + 1\nsum",
expected: "900",
},
Workload {
name: "return-cleanup",
source: "find = (items) ->\n try\n for n in items then if n == 73 then return n\n nil\n catch error\n 0\n finally\n 0\nsum = 0\ni = 0\nwhile i < 100\n sum = sum + find([1...100])\n i = i + 1\nsum",
expected: "7300",
},
];

fn usage() {
Expand Down
43 changes: 40 additions & 3 deletions tests/cli_tools.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -503,9 +503,46 @@ fn qbench_json_is_guarded_and_machine_readable() {
assert!(json.status.success());
let stdout = String::from_utf8_lossy(&json.stdout);
let lines: Vec<_> = stdout.lines().collect();
assert_eq!(lines.len(), 6);
assert!(stdout.contains("\"name\":\"stepped-string-iteration\""));
assert!(stdout.contains("\"name\":\"signed-by-iteration\""));
let expected_names = [
"loop-core",
"closures-and-ranges",
"map-spread",
"negative-indexing",
"stepped-string-iteration",
"signed-by-iteration",
"postfix-loops",
"array-slices",
"existence-tests",
"existential-assignment",
"name-updates",
"floor-modulo",
"bitwise",
"multiline-strings",
"string-iteration",
"string-escapes",
"string-indexing",
"multiline-collections",
"indented-maps",
"implicit-calls",
"execution-stats",
"constant-folding",
"bare-lambda",
"stepped-iteration",
"for-collection",
"postfix-comprehension",
"for-pattern-bindings",
"maps-and-control",
"soak-access",
"nested-destructuring",
"destructuring-rest",
"chained-comparisons",
"destructuring-parameters",
"return-cleanup",
];
assert_eq!(lines.len(), expected_names.len());
for name in expected_names {
assert_eq!(stdout.matches(&format!("\"name\":\"{name}\"")).count(), 1);
}
for line in lines {
assert!(line.starts_with('{') && line.ends_with('}'));
for field in [
Expand Down