A TypeScript/JavaScript library for carefully refactoring critical paths.
This is a port of GitHub's Ruby library Scientist, by @jbarnette, @jesseplusplus, @rick, @zerowidth and contributors. The API, the behavior and most of the documentation follow the original as closely as JavaScript allows; the differences are documented. Like the original, it is released under the MIT license.
It works in Node.js (20 or newer) and other modern JavaScript runtimes, ships ESM and CommonJS builds with type declarations, and has no runtime dependencies.
This port tracks the Ruby library at tag v1.6.5 (commit
504a396e987f655a21c6bf2ed57935aadaa40859).
The sources were read from main at commit
2cd5c16210a4f4c938f48395b2860ebe0ccf56e7
(2026-09-28). That commit differs from v1.6.5 only in CI and dependabot configuration, so lib/ and test/ are
identical. The ported test suite in test/ mirrors upstream's test/scientist/*_test.rb at that revision. When
syncing with a newer upstream release, compare lib/ from this tag onwards.
The package is published to npm as @mhvelplund/scientist.
Install it with your package manager:
npm install @mhvelplund/scientist
# or
pnpm add @mhvelplund/scientist
# or
yarn add @mhvelplund/scientistYou don't need to clone or build this repository to use the library; that is only needed to contribute.
TypeScript type declarations ship with the package (dist/index.d.mts for import, dist/index.d.cts for require),
so there is no separate @types/... package to install. They are resolved through the exports map and work with
moduleResolution set to nodenext, node16 or bundler.
Let's pretend you're changing the way you handle permissions in a large web app. Tests can help guide your refactoring, but you really want to compare the current and refactored behaviors under load.
import { Experiment, type Result, Scientist } from "@mhvelplund/scientist";
// 1. Decide how experiments are enabled and where results go.
class MyExperiment<T = unknown> extends Experiment<T> {
enabled(): boolean {
return true; // see docs/custom-experiments.md for ramping up
}
publish(result: Result<T>): void {
console.log(this.name, result.matched() ? "matched" : "mismatched");
}
}
Experiment.setDefault(MyExperiment);
// 2. Wrap the old code in `use` and the new code in `try`.
class MyWidget extends Scientist {
allows(user: User): boolean {
return this.science<boolean>("widget-permissions", (e) => {
e.use(() => model.checkUser(user).valid); // old way
e.try(() => user.can("read", model)); // new way
}); // returns the control value
}
}Without a base class, call Scientist.run (or the standalone run / science functions):
import { Scientist } from "@mhvelplund/scientist";
const allowed = Scientist.run<boolean>("widget-permissions", (e) => {
e.use(() => model.checkUser(user).valid);
e.try(() => user.can("read", model));
});Behaviors that return promises are supported with runAsync, scienceAsync and Experiment#runAsync; see
async experiments.
const { Experiment, Scientist } = require("@mhvelplund/scientist");
class MyExperiment extends Experiment {
enabled() {
return true;
}
publish(result) {
console.log(this.name, result.matched() ? "matched" : "mismatched");
}
}
Experiment.setDefault(MyExperiment);
const allowed = Scientist.run("widget-permissions", (e) => {
e.use(() => model.checkUser(user).valid);
e.try(() => user.can("read", model));
});Wrap a use callback around the code's original behavior, and wrap try around the new behavior. run always returns
whatever the use callback returns (or throws what it threw), but it does a bunch of stuff behind the scenes:
- It decides whether or not to run the
trycallback (enabled()andrunIf), - Randomizes the order in which
useandtrycallbacks are run, - Measures the wall time and CPU time of all behaviors, in seconds,
- Compares the result of
tryto the result ofuse, - Swallows and records errors thrown in the
trycallback, and - Publishes all this information.
The use callback is called the control. The try callback is called the candidate.
If you don't declare any try callbacks, none of the Scientist machinery is invoked and the control value is always
returned.
Note that the built-in default experiment (DefaultExperiment) is never enabled and publishes nothing, so nothing
interesting happens until you register your own experiment class with Experiment.setDefault.
The full guide lives in docs/index.md. It covers custom experiments, comparing results, context, cleaning, ignoring mismatches, publishing, testing, error handling, multiple candidates, async experiments, plain JavaScript usage, and the differences from the Ruby library.
This repository ships an agent skill that teaches AI coding agents (Claude Code, Cursor,
Codex, GitHub Copilot and others) how to use this library: registering an experiment class, choosing between run and
runAsync, writing a publisher, what is safe to experiment on, and the common mistakes. It lives in
skills/scientist-experiments/.
Install it with the skills CLI:
npx skills add mhvelplund/scientist.jsThe CLI asks which agents to install the skill for and whether to install it in the current project or globally. Pass
--skill scientist-experiments to pick the skill without the prompt, -g to install it globally, or -a claude-code
(for example) to target a specific agent.
Every tool this repository needs is pinned in mise.toml and installed by mise:
Node.js, the GitHub CLI (gh), EarthBuild (earth), actionlint, and, via uv,
commitizen (cz) and pre-commit. Install
mise by following the instructions at https://mise.jdx.dev, then from a checkout of this repository:
mise install # installs the pinned tools
mise run hooks # installs the git hooks (pre-commit, commit-msg, pre-push)
mise run ci # installs dependencies, lints, type-checks, tests, builds and smoke-testsThe containerized build (earth +ci, see Earthfile) also needs Docker or Podman; run earth bootstrap
once after installing.
Commit messages must follow Conventional Commits (feat: ..., fix: ...,
docs: ..., feat!: ... for breaking changes, and so on), because releases are versioned from them. The hooks in
.pre-commit-config.yaml check each commit message, and before every commit and every push
they run Biome and some basic file checks. Before a push they also check that every pushed commit message is
conventional. Use cz commit for an interactive prompt that writes a valid message.
Every task is defined in mise.toml and run with mise run <task>:
| Task | What it does | Runs |
|---|---|---|
install |
Install npm dependencies (skipped when the lockfile is unchanged) | npm ci |
lint |
Lint and check formatting with Biome | biome check . |
format |
Format all files with Biome | biome check --write . |
typecheck |
Type-check sources and tests | tsc --noEmit |
test |
Run the unit tests with coverage | vitest run --coverage |
build |
Build ESM + CJS bundles and type declarations into dist/ |
tsdown |
smoke |
Load the built package from CommonJS and ESM, and type-check its declarations | node test/smoke/smoke.{cjs,mjs}, tsc -p test/smoke |
pack |
Show what would be published to npm | npm pack --dry-run |
hooks |
Install the git hooks | pre-commit install |
bump |
Preview the next version and changelog commitizen would release | cz bump --dry-run --changelog |
clean |
Remove build and coverage output | rm -rf dist coverage |
ci |
Everything CI runs: lint, typecheck, test, build, smoke | depends on the tasks above |
The build tasks depend on install, so a fresh checkout only needs mise run <task>.
.github/workflows/ci.yml runs on every push to main and every pull request. It installs
the pinned tools with jdx/mise-action, then runs earth --ci +ci, which runs
the same steps as mise run ci in containers. If mise run ci passes locally, CI should pass too.
main is protected: it can only change through a pull request, and the ci check must pass before the pull request
can merge.
To use a released version, install it from npm. Only if you need to try unreleased changes from a checkout in another local project, either pack it into a tarball:
# in this repository
mise run build
npm pack # creates mhvelplund-scientist-<version>.tgz
# in the other project
npm install /path/to/scientist.js/mhvelplund-scientist-<version>.tgzor link it, so that rebuilding here is picked up there immediately:
# in this repository
mise run build
npm link
# in the other project
npm link @mhvelplund/scientistThe tarball is closer to what users get from the registry (only dist/ is included); npm link is more convenient
while iterating.
Releases are automatic. On every push to main,
.github/workflows/release.yml runs commitizen (configured in
.cz.toml) on the Conventional Commits since the last tag:
- If there are no
feat,fixor breaking-change commits, nothing happens. - Otherwise commitizen bumps
versioninpackage.jsonandpackage-lock.json, updatesCHANGELOG.md, and commits and tagsvX.Y.Zas thecommitizen-ciGitHub App. That app is the only actor allowed to push tomaindirectly. - The workflow builds the tarball with
earth +packand publishes it to npm with trusted publishing and provenance. No npm token is stored. - It creates a GitHub release for the tag with that version's changelog. While the version is
0.x, releases are marked as pre-releases.
While on 0.x, breaking changes bump the minor version. Never edit the version or CHANGELOG.md by hand. Run
mise run bump to preview what the next release would be.
MIT. See LICENSE. This project is a port of github/scientist, which is also MIT licensed.