CONTRIBUTING.md

Contributing to Reef

Thank you for helping improve Reef. Contributions can include bug reports, documentation, tests, examples, code, design proposals, and reviews.

Before you start

• Search the existing issues and pull requests before opening a new one.
• Keep each issue and pull request focused on one problem or proposal.
• For a bug, use the bug report form and include a reproducible example.
• For a change whose direction or scope is uncertain, open an issue before investing in an implementation.
• Never include credentials, private data, model-provider tokens, or other secrets in an issue, pull request, log, fixture, or prompt transcript.

Choose the right issue path

Use the structured template that matches the work:

• Bug report for reproducible incorrect behavior. Include the smallest complete reproduction and environment details.
• Performance report for a measurable latency, throughput, memory, utilization, or scalability regression. Include equivalent baseline and current measurements.
• Usage question when the documentation and open or closed issues do not answer a focused question.
• Feature proposal for a concrete user problem whose scope does not yet require a durable architecture decision.
• Experiment for a paper reproduction, benchmark, or empirical question with pinned models, workloads, baselines, metrics, and saved results.
• Example for a runnable user-facing recipe or reference deployment with a documented setup and expected result.
• RFC proposal when the change may affect public interfaces, persistence, trust boundaries, topology, recipe extension contracts, training backends, or project-wide policy.
• Maintainer task for scoped implementation, refactoring, cleanup, or project work that a maintainer has created or approved.
• Roadmap only for a maintainer-owned, time-bounded coordination issue.

Issue and RFC writing

Write for a reviewer deciding what to do next. Ordinary issues usually need only a few paragraphs; aim for roughly 300–500 words for a focused RFC. These are defaults, not limits: include detail when it changes the decision.

• Lead with the concrete problem and requested outcome or decision. Use one short example when it makes the proposed behavior clearer.
• Describe the smallest useful change and what existing mechanisms it reuses. Do not add frameworks, configuration, or requirements without a concrete need.
• State each point once. Cut repeated summaries, conversational history, speculative edge cases, and lists of unrelated non-goals.
• Use plain language and short paragraphs or bullets. Include implementation details only when needed to assess feasibility, compatibility, or a trade-off.
• Use the template to check coverage, not to generate length. Keep required answers brief; use “Not applicable” where appropriate. When writing outside the form, group related topics under a few headings while covering the applicable questions.
• Preserve material risks, compatibility constraints, and acceptance checks. Separate verified behavior from proposals and untested assumptions. Link supporting code, results, and related work instead of copying them into the body.

Before submitting, remove any sentence that neither helps explain the problem nor affects the decision or its verification. The concise proposal in #688 illustrates this style; it is not a required outline or an endorsement of its design.

Issue titles and classification

An issue title starts with exactly one controlled type prefix:

Prefix	Use it for
[Bug]	Reproducible incorrect behavior
[Feature]	A new capability, algorithm, or integration
[Performance]	Latency, throughput, memory, utilization, or scale
[Task]	Scoped refactoring, cleanup, documentation, CI, or project work
[Experiment]	A benchmark, reproduction, or empirical evaluation
[Example]	A runnable recipe, demo, or reference deployment
[RFC]	A durable architecture, interface, or policy decision
[Roadmap]	A quarterly or release-level coordination index
[Question]	A focused usage question

Write the rest of the title as a concise outcome, normally beginning with a verb: [Feature] Add W&B experiment tracking for training. Do not add a second bracketed tag for an area, project, status, or method. In particular, use [Roadmap] Reef 2026 Q3, not [Roadmap][Draft] ...; record Draft, Active, or Complete in the roadmap body.

Labels carry the independent classification dimensions. Maintainers may apply more than one area:* label when work crosses boundaries:

• area: core — core contracts and scenario lifecycle;
• area: service — service APIs, deployment, and CLI;
• area: training — training execution, runtimes, and algorithms;
• area: artifacts — artifact storage, metadata, retention, and publication;
• area: versioning — identity, lineage, activation, promotion, and rollback;
• area: observability — metrics, logs, traces, and experiment tracking;
• area: harness — harness integration, recipes, and evolution surfaces;
• area: examples — examples, recipes, and reference deployments;
• area: packaging — packages, containers, and vendored integrations;
• area: ci — automation, CI, and developer tooling; and
• area: docs — user and developer documentation.

Lifecycle (status:*), priority (priority:*), contribution, and accelerator-validation labels do not belong in the title.

Security vulnerabilities do not belong in public issues. Follow SECURITY.md and establish a private reporting channel before sending exploit details.

All new issues begin with status: needs-triage. Maintainers add an area and one lifecycle status after review. If a maintainer requests information, the issue may receive status: waiting-author; a new comment from the issue author returns it to triage automatically. help wanted means maintainers welcome an external contributor to propose a plan and request assignment. It is not an invitation to submit competing pull requests without coordination.

Changes that need an RFC

Write an RFC before implementing a change that:

• adds a new top-level package under reef;
• adds a shared training backend or changes the recipe extension contract;
• makes a significant or backwards-incompatible public interface change;
• changes a persisted format, wire contract, trust boundary, or service topology; or
• establishes a project-wide policy that will constrain later work.

Small bug fixes, documentation improvements, tests, internal refactors that preserve behavior, and implementation work for an accepted RFC do not normally need a new RFC.

Open an RFC issue and complete the full proposal in the issue body. The issue is the RFC and its decision record; do not add an RFC document to the repository. Keep material design changes, the maintainer decision, and implementation links on the issue. An RFC must be explicitly accepted there before its implementation is treated as approved project direction.

AI-assisted contributions

AI-assisted work is welcome when it is directed, understood, and verified by the human contributor. Do not submit autonomous or bulk-generated issues, pull requests, reviews, or comments.

When an AI tool makes a non-trivial contribution:

• disclose the tool and how it was used in the pull request;
• review and understand every submitted line and factual claim;
• reproduce the problem yourself instead of trusting a generated diagnosis;
• run the relevant checks and report their actual results;
• remove speculative fixes, unrelated cleanup, generated commentary, and unnecessary abstractions; and
• communicate with maintainers in your own words and remain responsible for follow-up review and maintenance.

AI assistance does not lower the bar for tests, documentation, compatibility, security, or long-term maintenance. Low-effort or unverifiable generated work may be closed without detailed review when reviewing it would cost more than reproducing or implementing the change directly.

Set up the repository

Follow the development guide to initialize submodules, create an environment, install dependencies, and enable pre-commit.

The usual local checks are:

pre-commit run --all-files
pytest tests/ -n 8

Run the suite in parallel, as CI does; pytest-xdist ships in the dev extra. Use -n 0 for a serial run when you diagnose a failure.

Keep the root READMEs synchronized

README.md and README.zh.md are one reviewed documentation pair. A change to either file must update the other when needed and preserve the same headings, lists, tables, link targets, and fenced code. After reviewing both languages, record their exact Git blob hashes and verify the pair:

python .github/scripts/check_readme_i18n.py --write
python .github/scripts/check_readme_i18n.py

The record in README.i18n.yaml makes any later one-sided edit fail local pre-commit checks and CI. The structural check does not judge translation quality, so its success does not replace human review of meaning and wording.

The full test suite needs the supported container environment and training dependencies. See the testing guide for focused commands and dependency details.

Understand the codebase

Use the codebase structure map to decide which package owns a change. Each package's __init__ docstring states the invariants inside its area; the map is the repository-wide routing guide.

When adding a recipe, training integration, runtime, surface, artifact type, service route, or harness adapter, follow the component playbooks. They list the implementation location, registration point, tests, configuration, and documentation expected in the same pull request.

Python coding style

Python changes should follow PEP 8, the Google Python Style Guide, and the principles below. The repository configuration in pyproject.toml is the authority when a general guide differs from Reef's mechanical style: Black formats code, isort orders imports, Ruff checks common errors and maintainability issues, and mypy checks the reef package. Do not hand-format code against these tools.

Write Pythonic code

• Prefer straightforward Python constructs and the standard library over custom abstractions. Code should make its control flow and data flow obvious.
• Use iterators, comprehensions, context managers, unpacking, and standard protocols when they improve readability. Use an ordinary loop when a comprehension would need complex conditions or side effects.
• Keep functions focused. Extract a helper when it gives a concept a useful name or removes meaningful duplication, not merely to shorten a function.
• Avoid clever metaprogramming, hidden global state, and surprising side effects. Make dependencies and state transitions explicit.
• Add type annotations to new or changed interfaces. Do not use Any to avoid describing a type unless the boundary is genuinely dynamic.
• Write docstrings for public modules, classes, functions, and methods when their purpose, contract, or failure modes are not clear from the signature. Comments should explain why a choice is necessary, not restate what the code does.

Use object-oriented design deliberately

• Use a class when data and behavior form a cohesive object with state, invariants, lifecycle, or a polymorphic contract. Prefer a function for a stateless transformation and a dataclass for a data-only value.
• Give each class one clear responsibility and keep its public surface small. Construct valid objects rather than relying on callers to set attributes in a particular order.
• Prefer composition and small abstract interfaces over deep inheritance hierarchies. Inheritance should represent a genuine substitutable relationship, not just reuse implementation.
• Encapsulate mutable state and expose intent-revealing operations. Do not add Java-style getters and setters when direct attribute access or a property is clearer.
• Keep I/O and framework integration at the edges so that core behavior can be tested with ordinary Python objects.

Define every type with mathematical rigor

A type is a mathematical object: a set of values, and the operations that make and read them. Define each new or changed class, dataclass, enum, and type alias with the rigor of a definition in mathematics. From the definition alone, a reader must be able to tell what the type can hold and what each operation does with it.

• Define the value set exactly. Give a membership condition: a value belongs to the type if and only if the condition is true. Name each field and its domain, and state each invariant between fields. For a collection, state the element type, whether order matters, whether duplicates are allowed, and whether it can be empty. Use set notation, such as R ⊆ S or refs(r) ⊆ R, when words are not exact.
• State the one purpose of the type: which operation makes its values, which operation reads them, and what a value means to that reader.
• Define each operation on the type by its inputs, its outputs, and the relation between them. For example: for each item i that make_samples(X) returns, sources(i) ⊆ records(X).
• Make the definition complete. Each enum member, each status, and each case of an operation has a defined meaning and a defined result. No value can reach a state that the definition does not cover.
• Make the definition consistent. Invariants must not contradict each other. A subclass is a subset of its base class: it satisfies every condition of the base definition and only adds conditions.
• Check the invariants when a value is made, in the constructor or in __post_init__. A caller must not be able to make a value that breaks them.
• Give each type one role. Split a type that has two jobs into two types. Do not use a base class as a loose category of different things.
• If two values of the type can hold the same member, state what happens then, for example which value consumes a shared record.

Put the definition in the docstring of the type. For example:

@dataclass(frozen=True)
class ReportGroup:
    """Reports that must train in the same job.

    Values: a tuple ``reports`` belongs to the type if and only if it is
    non-empty, all its reports are of one scenario and have the same group key,
    it is in arrival order, and no report ID occurs twice.
    Purpose: the record stage makes it when the group is complete. The pipeline
    reserves all of ``reports`` for one job's data, or none of them.
    """

A docstring such as "Records that are used together" is not a definition. It gives no membership condition, so a reader cannot tell which values the type can hold or which code reads them.

Avoid dynamic design shortcuts

• Do not use TYPE_CHECKING. Imports needed by annotations must also be valid at runtime. Resolve cycles by moving shared contracts to a lower-level module, correcting the dependency direction, or using a local runtime import at the integration boundary.
• Do not use typing.Protocol, typing_extensions.Protocol, or runtime_checkable. Define abstract base classes and inherit them explicitly. The Python design check enforces this across all first-party Python files, including reef/, recipes/, tests/, tutorials/, docker/, docs/, .github/scripts/, and root files. It excludes third-party code, local dependency/build trees, vendored benchmark inputs, published result programs, and golden fixtures using the paths in .github/scripts/check_python_design.py. Protocol findings cannot be exempted through the design baseline. The existing TYPE_CHECKING and Callable checks keep their reef/, recipes/, and tests/ scope.
• Do not model long-lived behavior as Callable constructor arguments, callable-valued fields, or containers of callbacks. Define an abstract base class with meaningful methods or a cohesive class so the contract, state, and lifecycle are explicit.
• A single short-lived callback can be appropriate for an algorithm, decorator, or standard-library adapter. A function that needs multiple callbacks is a sign that those operations belong to one interface.
• Do not use dynamic dispatch through getattr, monkey-patching, or runtime inspection when an explicit interface or ordinary polymorphism expresses the same design. Boundary adapters for third-party frameworks must keep such behavior local and document why it is necessary.
• Do not hide a design-policy violation with a type alias or lint suppression. Any rare exception must be narrowly scoped, justified in review, and recorded in the design-check baseline.

Choose names for readers

• Use snake_case for modules, functions, methods, and variables; PascalCase for classes; and UPPER_SNAKE_CASE for constants.
• Name classes with concrete noun phrases and functions with verb phrases. Predicate names should read as questions, such as is_ready, has_capacity, or can_retry.
• Prefer domain words and complete, familiar terms over internal shorthand. Avoid vague names such as data, info, obj, manager, helper, or utils when a more specific name exists.
• Include units or representation when ambiguity could cause a bug, for example timeout_seconds, token_count, or checkpoint_path.
• Keep terminology consistent across code, configuration, logs, and documentation. A single concept should have a single name.
• Short conventional names are fine in a small scope (i in an index loop, f for a local file handle). Do not carry them across a larger scope.

For example, prefer:

def select_ready_workers(workers: Iterable[Worker]) -> list[Worker]:
    return [worker for worker in workers if worker.is_ready]

over names that hide the domain and intent:

def process(data):
    return [x for x in data if x.status]

Automated checks and review

The automated checks enforce the parts of this guide that can be evaluated reliably:

• Black and isort enforce formatting and import order.
• Ruff's Pythonic and correctness rules catch unnecessarily complex constructs, error-prone patterns, and common performance problems.
• Ruff's pep8-naming rules enforce the naming forms above. Narrow exceptions for established public APIs and mathematical notation are documented in pyproject.toml.
• mypy checks type consistency in the reef package.
• The Python design-policy check rejects Protocol, runtime_checkable, TYPE_CHECKING, and new Callable-based object state or callback bundles. Its baseline identifies existing migration debt for Callable patterns; Protocol and TYPE_CHECKING findings cannot be baselined.

Automation cannot determine whether a class is the right abstraction, whether a type has a rigorous definition, whether an identifier uses the clearest domain term, or whether an interface has one cohesive responsibility. Authors and reviewers must evaluate those design and readability requirements during review. Do not add a lint exception merely to silence a warning; keep it narrow and explain why the general rule does not fit.

Before requesting review for Python changes, run:

pre-commit run --all-files
python -m mypy
pytest tests/ -n 8

Make a pull request

Before requesting review:

• keep the diff as small as practical and remove unrelated changes;
• explain what changed, why it is needed, and how it was verified;
• link the relevant issue or RFC when one exists;
• add or update tests for behavior changes;
• add a contract test for a public interface change;
• update user, operator, or developer documentation affected by the change;
• run the relevant local checks and report the commands and results.

Use a draft pull request when the design or implementation is not ready for acceptance. Do not mix a functional change with drive-by formatting, generated rewrites, or unrelated cleanup.

Draft pull requests run lint, type checks, and static Dockerfile checks only. Once ready, each update automatically runs the source, sandbox, and installed- wheel suites on Python 3.12, including the combined coverage check. Documentation builds and real-harness smoke tests run automatically when their files change. Run relevant checks locally and batch each round of review fixes before pushing.

Before merging, the required checks must pass on the latest revision: lint, test (3.12), and package (3.12). Reef supports Python 3.12 only, so every run tests the same version. A new push cancels older runs. To retry a failed check, any contributor with repository write access can rerun the latest ci run of the pull request:

gh run rerun RUN_ID --repo Human-Agent-Society/reef

A rerun fails if the pull request is closed, is a draft, or has newer commits. Authors without repository write access can ask a collaborator to rerun it.

Review and acceptance

GitHub requests reviews from the code owners of the affected areas. The maintenance model defines Maintainer, Code Owner, and Area Reviewer responsibilities and the merge process. Reviewers may ask for changes to correctness, interfaces, tests, documentation, compatibility, operability, or scope. Authors are expected to respond to substantive comments and to say when a request is unclear or when they disagree.

A pull request is ready to merge only when required reviews are complete, required checks pass, and no blocking discussion remains. Passing CI does not guarantee acceptance: maintainers also consider project direction, long-term maintenance cost, compatibility, and reviewer capacity.

If a review has gone quiet, a concise and courteous reminder on the pull request is welcome. The project cannot guarantee a response or merge timeline. Non-draft pull requests are marked stale after 60 inactive days and close 21 days later; issues are marked after 90 inactive days and close 30 days later. Maintainers may apply status: keep-open or status: blocked when inactivity is expected. Closed work can be reopened or resubmitted when it becomes current again.

License

By contributing, you agree that your contribution may be distributed under the repository's Apache License 2.0.